-- Logs begin at Wed 2022-10-26 10:28:04 UTC, end at Wed 2022-10-26 11:43:42 UTC. -- Oct 26 10:59:06.297212 dv-cinder-815582-18-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 26 10:59:06.304449 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 26 10:59:06 2022] *** Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: nodename: dv-cinder-815582-18-iscsi Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: machine: x86_64 Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: clock source: unix Oct 26 10:59:06.306441 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: pcre jit disabled Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: detected number of CPU cores: 4 Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: current working directory: / Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: detected binary path: /usr/bin/uwsgi-core Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: your processes number limit is 47680 Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: your memory page size is 4096 bytes Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: detected max file descriptor number: 2048 Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: lock engine: pthread robust mutexes Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: thunder lock: enabled Oct 26 10:59:06.306943 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 26 10:59:06.309197 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 26 10:59:06.346914 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: Python main interpreter initialized at 0x55d7cbe959c0 Oct 26 10:59:06.346914 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: python threads support enabled Oct 26 10:59:06.346914 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: your server socket listen backlog is limited to 100 connections Oct 26 10:59:06.346914 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: your mercy for graceful operations on workers is 90 seconds Oct 26 10:59:06.347932 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: mapped 403077 bytes (393 KB) for 2 cores Oct 26 10:59:06.347932 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: *** Operational MODE: preforking *** Oct 26 10:59:06.348207 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: *** uWSGI is running in multiple interpreter mode *** Oct 26 10:59:06.348207 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: spawned uWSGI master process (pid: 81732) Oct 26 10:59:06.348413 dv-cinder-815582-18-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 26 10:59:06.354843 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: spawned uWSGI worker 1 (pid: 81744, cores: 1) Oct 26 10:59:06.354843 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: spawned uWSGI worker 2 (pid: 81745, cores: 1) Oct 26 10:59:06.354843 dv-cinder-815582-18-iscsi devstack@keystone.service[81732]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 26 10:59:10.671488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:59:10.671488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: from cryptography.utils import int_from_bytes Oct 26 10:59:10.672278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:59:10.672278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: from cryptography.utils import int_from_bytes Oct 26 10:59:10.672692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:59:10.672692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: from cryptography.utils import int_from_bytes Oct 26 10:59:10.672814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 26 10:59:10.672814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: from cryptography.utils import int_from_bytes Oct 26 10:59:10.722470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 26 10:59:10.722641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 26 10:59:10.722710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 26 10:59:10.722789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 26 10:59:10.722888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] command line args: [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 26 10:59:10.722950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] command line args: [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 26 10:59:10.723185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 26 10:59:10.723269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 26 10:59:10.723336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ================================================================================ {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 26 10:59:10.723407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ================================================================================ {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 26 10:59:10.723577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] admin_token = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.723663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] admin_token = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.723798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] config_dir = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.723875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] config_dir = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] config_source = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] config_source = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] debug = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] debug = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.724521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.725884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.725884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_config_append = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_config_append = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_dir = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_dir = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.726992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_file = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_file = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.727832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.728128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.728235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.728323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.728433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.728660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.730569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.730569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.730901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.731977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] publish_errors = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.732896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] publish_errors = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] standard_threads = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] use_journal = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] use_json = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.733980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] use_stderr = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] standard_threads = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] use_syslog = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.734568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] use_journal = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.734769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] use_json = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.734945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] use_stderr = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.734945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] use_syslog = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.735136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 26 10:59:10.735286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.735930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.736939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.737113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.737201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.737361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.737932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.738942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.739947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.740896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.741993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.742985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.743909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.744897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.745979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.746951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.747893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.748807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.749374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.749511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.749715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.749821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.749918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.750998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.751069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.754967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.755926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.756956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.757984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.758937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.759976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.external = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.760991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.password = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.external = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] auth.token = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.761935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.password = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] auth.token = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.762927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.763981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.764935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.765928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.766867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.767969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.768953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.769918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.770949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.771863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.772990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.773946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.774959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.775944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.776986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.777951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.778925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.779867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.780912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.781946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.782984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.783932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.784983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.785930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.786791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.788796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.788876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.788938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.788998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.789943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.790964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791406 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.791956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.792922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.793977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] role.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] role.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.794962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] role.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] role.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.795959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.796945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.797916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.798804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.799756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.800439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.800527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.800724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.800803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.800975 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.801997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.802985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.803952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.804885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.805854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.806868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.807823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.connection = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808239 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.808988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.connection = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.809938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.810971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.811923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 26 10:59:10.812586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.812801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.813040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.813273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 26 10:59:10.813441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81744) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 26 10:59:10.842452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.842680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.842836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.842943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.843081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.843307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.851874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.852206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.881962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.882138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.882226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.882371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.943926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.944407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.995994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.996221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.996401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:10.996641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.157765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.157980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.158858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.159033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.159178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.188759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.188961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.189953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.190072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.190157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.190287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.201105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.201529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.214386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.214641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.214730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.214983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.215950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81744) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.216084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81745) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 26 10:59:11.233330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.233482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.233567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:59:11.233698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:59:11.233868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.234048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.234131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:59:11.234265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 26 10:59:11.234357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.234568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.234892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.235157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.235253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.235415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.235495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.235763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.244768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.245092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.245360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.245619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.245705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.245907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.246185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.246471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.252543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.253556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.255429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.255715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.255998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.256279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.256463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.256546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.256754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.256837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.257955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.258208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.258346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.258465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.258840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.259126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.259404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.281637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.282701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.283218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.283512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.284191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.285023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.290415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.290701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.291007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.291451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.291744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.292033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.297381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.297670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.298422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.298713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.301860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.302186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.307821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.307821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.307996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.308064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.308620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.319479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.319757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.320489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.320776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.323571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.324638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.326864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.327161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.327910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.328215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.332405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.336435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.336535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.336602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.338810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.339129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.339415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.339511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.339740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.339824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.340901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.356516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.356803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.357072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.357238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.357322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.357533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.357807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.358074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.364185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.365035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.365744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.366570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.367282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.367567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.368284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.368576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.370165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.370452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.370971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.371254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.371345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.371535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.371635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.371927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.372217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.372522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.382544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.382850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.383231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.383546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.383662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.383823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.383950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.384262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.384357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.384489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.384636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.384935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.385221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.385504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.400804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.401723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.403698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.404677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.406586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.407751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.408110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.409183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.410258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.410551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.410851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.411365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.411667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.411956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.417169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.418343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.418655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.419866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.421702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.422523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.422523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.422718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.422916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.423213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.423505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.423790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.430316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.430602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.430904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.431626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.431922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.432208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.438837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81744) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.439143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.439422 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.439770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.439998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81745) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 26 10:59:11.440084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.440959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.441970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81744) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.442797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.443116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.443395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.443661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81745) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 26 10:59:11.476598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d7cbe959c0 pid: 81744 (default app) Oct 26 10:59:11.477603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d7cbe959c0 pid: 81745 (default app) Oct 26 10:59:11.479893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5b0b-4a4d-4c86-bef0-fb0e6a8dff88 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:11.480178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5b0b-4a4d-4c86-bef0-fb0e6a8dff88 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:11.480449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c5b0b-4a4d-4c86-bef0-fb0e6a8dff88 None None] PATH_INFO: `/v3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:11.484609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Oct 26 10:59:11 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:19.038047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:19.038525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:19.038525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:19.042793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:19.048144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:19.048542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:19.048681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:19.120287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: INFO dbcounter [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] Registered counter for database keystone Oct 26 10:59:19.139117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG oslo_db.sqlalchemy.engines [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81744) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 26 10:59:19.145320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writer thread running {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 26 10:59:19.320290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG passlib.handlers.bcrypt [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81744) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 26 10:59:19.320599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG passlib.handlers.bcrypt [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81744) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 26 10:59:19.378134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:19.379824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:19.503860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 26 10:59:19.934478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:20.005322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: INFO dbcounter [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] Registered counter for database keystone Oct 26 10:59:20.027118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG oslo_db.sqlalchemy.engines [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81745) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 26 10:59:20.033264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writer thread running {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 26 10:59:20.151618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86d814-c003-450f-a6bb-80dbadb808eb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:20.155909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:20.156533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:20.156813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:20.157086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:20.165274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:20.216404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 26 10:59:20.216836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d89b0-ba54-43f5-aaa8-3e3b32e75265 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:20.226043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Oct 26 10:59:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 10:59:20.232049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:20.248940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b63d0b-708c-4f7b-8350-198d7b9a4721 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:20.254062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:20.255209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:20.255540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:20.255771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] PATH_INFO: `/v3/projects/6d130249a55c4bbea044091c5d05b944` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:20.259608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d130249a55c4bbea044091c5d05b944)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:20.309955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING keystone.common.rbac_enforcer.enforcer [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 26 10:59:20.310298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28155f8d-1102-4ca8-b8f6-43632ee11ef2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:20.315356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 10:59:20 2022] GET /identity/v3/projects/6d130249a55c4bbea044091c5d05b944 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:22.431598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afc44-8aeb-4979-a3ac-9a48dbdd7fe0 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:22.432108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afc44-8aeb-4979-a3ac-9a48dbdd7fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:22.432188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51afc44-8aeb-4979-a3ac-9a48dbdd7fe0 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:22.436216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:22.442395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:22.442833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:22.442971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:22.463049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:22.464539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:22.472267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 10:59:22.744920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51afc44-8aeb-4979-a3ac-9a48dbdd7fe0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:22.757468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51afc44-8aeb-4979-a3ac-9a48dbdd7fe0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:22.761644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:22.762242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:22.762527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:22.762804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:22.770556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:22.772403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f16506-fbf1-41e2-8de0-fc46fbc906ea admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:22.781241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Oct 26 10:59:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 10:59:22.786654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:22.795532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf85037-3902-48e5-98cf-9ae82456696b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:22.800221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:22.800839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:22.801116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:22.801386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] PATH_INFO: `/v3/users/e3c50bbb7acb4f91ac247acc31363261` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:22.802675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] RBAC: Authorizing `identity:get_user(user_id=e3c50bbb7acb4f91ac247acc31363261)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:22.804499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574814a-c944-475e-a418-018d2aef54c9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:22.805993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Oct 26 10:59:22 2022] GET /identity/v3/users/e3c50bbb7acb4f91ac247acc31363261 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:24.974165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3461d-4ae1-4646-90aa-d9c4cabdc723 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:24.974741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3461d-4ae1-4646-90aa-d9c4cabdc723 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:24.974741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3461d-4ae1-4646-90aa-d9c4cabdc723 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:24.976058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:24.981194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:24.981628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:24.981721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.003976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:25.003976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:25.051629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26aeb4-8164-445f-826d-79872611d775 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.051919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26aeb4-8164-445f-826d-79872611d775 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.052217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26aeb4-8164-445f-826d-79872611d775 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.053532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:25.058777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.059091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.059362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.083176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:25.101129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG passlib.handlers.bcrypt [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81745) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 26 10:59:25.101512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG passlib.handlers.bcrypt [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81745) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 26 10:59:25.150507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:25.152074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:25.247477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:25.352756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.377139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:25.454539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fc541-fffb-49e0-a680-fad68aeb7fe9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.488699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.489326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.489604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.489876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.500588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:25.509795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:25.510949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:25.511988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:25.513295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.516630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.596622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:25.614068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd2694-a286-4cdf-b4ac-32165eef66cf None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.614223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733b4d2-80cc-45af-b7ef-22240b1d5f01 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.616349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:25.648663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.649271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.649551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.649823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.650483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:25.650788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:25.650900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.651731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.652011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.652194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:25.652279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.653215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:25.654609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:25.664148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:25.669596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.703697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.sql.core [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:59:25.703697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:59:25.703697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [parameters: {'id': '2c8a9f0550f44b1392c5734e16eae625', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:59:25.703697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81745) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:59:25.721653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 10:59:25.722707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application yield resource Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise self.value Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application session.commit() Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 10:59:25.723795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.flush() Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application compat.raise_( Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 10:59:25.724673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 10:59:25.725563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 10:59:25.726316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [parameters: {'id': '2c8a9f0550f44b1392c5734e16eae625', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 10:59:25.727553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:25.728468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 8/16] 10.222.0.42 () {60 vars in 1137 bytes} [Wed Oct 26 10:59:25 2022] POST /identity/v3/domains => generated 190 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:25.729572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.771382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d98e9-2d2c-4e26-b785-b952be808375 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.805259 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.805885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.806167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.806445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] PATH_INFO: `/v3/users/e3c50bbb7acb4f91ac247acc31363261` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.807620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] RBAC: Authorizing `identity:get_user(user_id=e3c50bbb7acb4f91ac247acc31363261)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:25.809079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:25.810649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/users/e3c50bbb7acb4f91ac247acc31363261 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:25.816750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.830419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:25.831592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6065360-f62b-4de6-b915-8f9efe7b60f4 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.866446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.867260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.867545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.867821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.885311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:25.886774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:59:25.902376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:25.903452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:59:25.904638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:59:25.904638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:59:25.904638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:25.904638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:25.909725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:25.919204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b3377-3268-48c7-848d-f1f0527609bd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:25.934326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:25.953612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:25.954235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:25.954512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:25.954809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:25.955922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:25.957363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:25.959097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:25.964517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:26.011593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad84b4-71aa-4747-8e47-3487f4787e5a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:26.042703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:26.045083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:26.045685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:26.045963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:26.046233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:26.046896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:26.047200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:26.049257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239c75a-7a87-43b9-8b5e-c578120b450b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:26.062307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742938e-04f9-402d-aa43-e3f30477c8b8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:26.065963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 10:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:26.098369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:26.099092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:26.099387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:26.099662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] PATH_INFO: `/v3/domains/default/users/e3c50bbb7acb4f91ac247acc31363261/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:26.101675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e3c50bbb7acb4f91ac247acc31363261, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:26.103180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba784a-89e4-4f27-99c2-19203ff3b5c0 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:26.300672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 11/21] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Oct 26 10:59:25 2022] PUT /identity/v3/domains/default/users/e3c50bbb7acb4f91ac247acc31363261/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:59:28.599859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899a1e-49cc-4878-9f33-40d5e46aefc1 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:28.600381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899a1e-49cc-4878-9f33-40d5e46aefc1 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:28.600381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5899a1e-49cc-4878-9f33-40d5e46aefc1 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:28.601771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:28.620200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc995d-71bd-41bf-83e8-e187a8696480 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:28.620703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc995d-71bd-41bf-83e8-e187a8696480 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:28.620775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc995d-71bd-41bf-83e8-e187a8696480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:28.648189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-01bc995d-71bd-41bf-83e8-e187a8696480 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:28.649715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-01bc995d-71bd-41bf-83e8-e187a8696480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:28.744361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:28.835220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48d5b-7e23-48e3-bc1e-4862dfede57a None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:28.835565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48d5b-7e23-48e3-bc1e-4862dfede57a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:28.835769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48d5b-7e23-48e3-bc1e-4862dfede57a None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:28.837094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:28.842441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb149b7-76aa-4dd7-98b5-384e406998da None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:28.842733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb149b7-76aa-4dd7-98b5-384e406998da None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:28.843032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb149b7-76aa-4dd7-98b5-384e406998da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:28.871252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8eb149b7-76aa-4dd7-98b5-384e406998da None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:28.872477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8eb149b7-76aa-4dd7-98b5-384e406998da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:28.972011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:28.976602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:29.154286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64fbd5-e5bd-4d81-b7d7-533f77732336 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.154462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64fbd5-e5bd-4d81-b7d7-533f77732336 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.154738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64fbd5-e5bd-4d81-b7d7-533f77732336 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.156288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 13/26] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:29.163433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867232fa-4eb5-4621-9fac-e829c60ea3a0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.163914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867232fa-4eb5-4621-9fac-e829c60ea3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.164033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867232fa-4eb5-4621-9fac-e829c60ea3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.185241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64fbd5-e5bd-4d81-b7d7-533f77732336 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:29.211982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-867232fa-4eb5-4621-9fac-e829c60ea3a0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:29.213716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-867232fa-4eb5-4621-9fac-e829c60ea3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:29.310282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:29.334746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64fbd5-e5bd-4d81-b7d7-533f77732336 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:29.354270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:29.355641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef53f8-252e-4af9-b57f-a1852675b123 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.355931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef53f8-252e-4af9-b57f-a1852675b123 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.356206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef53f8-252e-4af9-b57f-a1852675b123 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.360509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 15/28] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:29.368119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.368407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.368683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.372400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:29.373243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.373800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.374299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.385122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:29.386889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 10:59:29.402865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:29.403727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 10:59:29.404594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 10:59:29.404594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 10:59:29.404594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:29.406365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:59:29.406770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:29.408549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:29.415894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:29.539821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 16/30] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:29.540652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:29.574374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da04a3b-e710-41ab-80b7-216c35f6f7d2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:29.629620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:29.639255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:29.639861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.640135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.640402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.641205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:29.652933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:29.652933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:29.655572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:29.667698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 10:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:29.675173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:29.706029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17acd4-a9db-4c0b-9f79-bc472db97c4e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:29.754737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:29.756046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.756326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.756595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.757263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:29.760452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:29.824766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396879c9-1904-4dc8-86ae-707855d968e1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:29.861100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:29.861764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:29.862043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:29.862310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:29.863055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:29.864501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:29.894335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 17/32] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Oct 26 10:59:29 2022] POST /identity/v3/roles => generated 219 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:29.895327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.082329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 16/33] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 10:59:29 2022] POST /identity/v3/projects => generated 301 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:59:30.083369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.101897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982270a8-37e7-42d9-9413-1a2b60ba70eb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:30.143431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:30.183733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:30.184378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.184665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.185012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.185941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:30.187167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:30.206050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:30.264840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae060-4955-4eb9-adbe-bcf4242255ce None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:30.308576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 18/34] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Oct 26 10:59:29 2022] POST /identity/v3/roles => generated 225 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:30.310526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:30.311265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.311548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.311823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.328177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-a156c986-7d7e-4e16-9487-6ba6e756e821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:30.340078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:30.341209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:30.341980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 17/35] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:30.343366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.449301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de89e-5383-490c-8544-701e95a64c8b None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.449933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de89e-5383-490c-8544-701e95a64c8b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.450403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55de89e-5383-490c-8544-701e95a64c8b None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.452071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 18/36] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:30.457814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ec21-986f-4a25-aa1d-dc5387af1037 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.458437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ec21-986f-4a25-aa1d-dc5387af1037 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.458858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ec21-986f-4a25-aa1d-dc5387af1037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.487516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057f5e0-4ec0-4c77-b1f7-444db6bf69cd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:30.488399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-cc15ec21-986f-4a25-aa1d-dc5387af1037 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:30.490317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cc15ec21-986f-4a25-aa1d-dc5387af1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:30.536804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:30.537979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.539390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.539829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.540661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:30.541119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:30.542706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd2fbe-8754-47aa-90a8-d0e12d7d0c22 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:30.554069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 19/37] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:30.555363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574066c4-124a-42c0-8af1-1ac06b61c6f5 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.555656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574066c4-124a-42c0-8af1-1ac06b61c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.555927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574066c4-124a-42c0-8af1-1ac06b61c6f5 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.557188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 20/38] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:30.561496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574066c4-124a-42c0-8af1-1ac06b61c6f5 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.584495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 19/39] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:30.585969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.586262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.586541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.611707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:30.613046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:30.621436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:30.643083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:30.684307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574066c4-124a-42c0-8af1-1ac06b61c6f5 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:30.684882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 20/40] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:30.718751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:30.719403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.719682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.720508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] PATH_INFO: `/v3/users/e3c50bbb7acb4f91ac247acc31363261` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.721622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] RBAC: Authorizing `identity:get_user(user_id=e3c50bbb7acb4f91ac247acc31363261)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:30.723141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:30.724919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 21/41] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:30 2022] GET /identity/v3/users/e3c50bbb7acb4f91ac247acc31363261 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:30.730545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.838172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3520c-1984-4d54-b84e-2e9555949cd5 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:30.865420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.872229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:30.872862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:30.873141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:30.873411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:30.874527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:30.876159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:30.877715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 21/42] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:30.884071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:30.920310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:30.993915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3981-9e90-4c67-b23c-5415bb1454a9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:31.028761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:31.029371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:31.029654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:31.029929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:31.030657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.domain.id=default, user.id=e3c50bbb7acb4f91ac247acc31363261) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:31.030997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:31.033108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:31.041510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 22/43] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Oct 26 10:59:30 2022] GET /identity/v3/role_assignments?user.id=e3c50bbb7acb4f91ac247acc31363261&scope.domain.id=default&role.id=27a106926ca14908b7e268013b8452a2 => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:31.042293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:31.099479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f0522-063b-441b-8bd2-b1a7f334e310 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:31.112551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:31.138986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:31.139653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:31.139934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:31.140211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:31.140902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:31.143029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1b6cd-701c-4134-a461-263f1b2c22fc None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:31.147252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:31.153903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c769ef-cea5-438b-90f5-8226d6a2be8d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:31.190202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:31.190882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:31.191167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:31.191444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:31.192596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:31.194072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:31.195751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 23/44] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:31.201803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:31.231026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Oct 26 10:59:30 2022] POST /identity/v3/roles => generated 223 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:59:31.318404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cdb7b-2d72-48d6-a155-6985174c66d9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:31.353797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:31.354421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:31.354704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:31.355009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:31.355672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:31.357130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413fc15-e4ea-48b4-9850-efe97746ddae None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:31.452018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 10:59:31 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:59:33.576514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f9fa-f98a-4393-9f93-25728eb80272 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:33.577143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f9fa-f98a-4393-9f93-25728eb80272 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:33.577143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68f9fa-f98a-4393-9f93-25728eb80272 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:33.578437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:33.583616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:33.584014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:33.584173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:33.602370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:33.603569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:33.614485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:33.667710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:33.934367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68f9fa-f98a-4393-9f93-25728eb80272 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:34.032955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68f9fa-f98a-4393-9f93-25728eb80272 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:34.065791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:34.066407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:34.066691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:34.066997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:34.068110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:34.069549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f450c11-33fb-454b-9443-249b003c7215 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:34.071119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:34.076543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:34.172168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efda9aa-d65c-4966-a237-67d4a2e6304c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:34.307266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:34.307779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:34.308061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:34.308341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:34.308991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:34.310429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc251e5f-290f-4c13-b337-1ed1358f3b30 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:34.395100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:34.401482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 26/50] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Oct 26 10:59:34 2022] POST /identity/v3/projects => generated 298 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 10:59:36.536266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be884b7-87e1-45a3-80a5-da0b0c0f1bd0 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:36.536802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be884b7-87e1-45a3-80a5-da0b0c0f1bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:36.536802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be884b7-87e1-45a3-80a5-da0b0c0f1bd0 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:36.538152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 25/51] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:36.543403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:36.543789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:36.543953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:36.561412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:36.562592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:36.625911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 27/52] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:36.892143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be884b7-87e1-45a3-80a5-da0b0c0f1bd0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:36.932662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:36.991528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be884b7-87e1-45a3-80a5-da0b0c0f1bd0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:37.024284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:37.024904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:37.025186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:37.025460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:37.026550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:37.028011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbee1f9-c46c-423e-982a-a963b9ecf9ba None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:37.029551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:37.034973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:37.129376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f149d1e-cf17-423a-bea8-f5944ed43e8f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:37.153960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:37.161574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:37.162170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:37.162451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:37.166414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:37.166414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:37.166414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd4494-03fa-4b74-8167-d8dc68161f5a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:37.256846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 28/54] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 10:59:37 2022] POST /identity/v3/users => generated 275 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:40.439999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94840b1-8de1-4a4e-bc82-423f58859da8 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.440520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94840b1-8de1-4a4e-bc82-423f58859da8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.440595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94840b1-8de1-4a4e-bc82-423f58859da8 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.441944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 27/55] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:40.452956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d43c-a1a6-4ec1-85f7-d1cf65e3c1f4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.453848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d43c-a1a6-4ec1-85f7-d1cf65e3c1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.454267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91d43c-a1a6-4ec1-85f7-d1cf65e3c1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.480396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2f91d43c-a1a6-4ec1-85f7-d1cf65e3c1f4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:40.481801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2f91d43c-a1a6-4ec1-85f7-d1cf65e3c1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:40.560382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d646ec-534f-4c6b-b8ac-6ffb532d170d None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.560950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d646ec-534f-4c6b-b8ac-6ffb532d170d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.561426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d646ec-534f-4c6b-b8ac-6ffb532d170d None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.564464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 28/56] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:40.569141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd460-28ec-4ca2-88db-184d6b4aee39 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.569435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd460-28ec-4ca2-88db-184d6b4aee39 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.569715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecd460-28ec-4ca2-88db-184d6b4aee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.578391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 29/57] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:40.596332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-97ecd460-28ec-4ca2-88db-184d6b4aee39 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:40.597540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-97ecd460-28ec-4ca2-88db-184d6b4aee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:40.693351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 29/58] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:40.914252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823fdd-b332-4045-b1bd-8486fbf2ef26 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.914571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823fdd-b332-4045-b1bd-8486fbf2ef26 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.914831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc823fdd-b332-4045-b1bd-8486fbf2ef26 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.916219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 30/59] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:40.919633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32798995-6065-4df1-adb7-c70d09028a9d None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.920163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32798995-6065-4df1-adb7-c70d09028a9d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.920641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32798995-6065-4df1-adb7-c70d09028a9d None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.922194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 30/60] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:40.926603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.927112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.927398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.927901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:40.928402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:40.928896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:40.955185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:40.956691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:40.967798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:40.968996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:41.027509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.046102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 31/61] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:41.069128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.093082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 31/62] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:41.119057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.120221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.174446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a69ab-2c6b-4e3b-87e3-4cfcfd85d1cb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.218234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.218939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.219229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.219509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.244557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:41.250421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:41.251462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:41.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 32/63] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:59:41.263292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.326693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c358cf-7f9f-4675-84cf-bc2578938b7b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.369420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.370061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.370348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.370628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.381096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:41.385938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:41.386790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:41.387617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 32/64] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:41.388260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.415420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edb54e-9b43-4972-b14c-17d96486af63 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.417684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.423602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.448172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.448781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.449063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.449380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.450001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:41.450307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:41.451735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04724ec-e84d-4a76-83a4-dd488b1261c8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:41.463361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 33/65] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:41.464530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c71e-621f-47ea-b428-bc7dda74e617 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.464813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c71e-621f-47ea-b428-bc7dda74e617 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.465159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c71e-621f-47ea-b428-bc7dda74e617 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.466465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 34/66] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:41.471666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c71e-621f-47ea-b428-bc7dda74e617 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.498649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bea40-6abc-4f3d-82d4-adb84949c7ac None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.532610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.533814 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.534097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.534371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.544167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:41.549011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:41.549927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 33/67] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:41.550896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.572990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c71e-621f-47ea-b428-bc7dda74e617 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.606581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.607223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.607506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.607782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.608443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:41.608749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:41.610136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:41.618470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 35/68] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:41.619476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.635919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.648002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.651163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d3b5d-cf64-475e-ac9d-5bc04928395a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.684507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.685326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.685612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.685889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.696293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-344639eb-e83d-44c6-b2fa-ddbc614bca35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:41.701108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:41.701934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:41.702699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:41.703810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.703810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.704032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.719015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e041-3f1b-48ab-bf05-9c6dd6d43244 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.722452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:41.723096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:41.755098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.755729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.756006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.756280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.771972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:41.773431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:41.775108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 36/70] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:41.775880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.789206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 35/71] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:41.789976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.866402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.867290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:41.883043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e74c6-807a-4597-97cd-b17bc3f4510f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.888300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520e5a1-d28f-43fb-b3e1-87b09b3b2568 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:41.917106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.917717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.918001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.918274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.918971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:41.919303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:41.920701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:41.924297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:41.924935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:41.925218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:41.925494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] PATH_INFO: `/v3/users/e3c50bbb7acb4f91ac247acc31363261` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:41.926633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] RBAC: Authorizing `identity:get_user(user_id=e3c50bbb7acb4f91ac247acc31363261)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:41.928132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:41.929553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 37/72] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:41.934153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 36/73] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/users/e3c50bbb7acb4f91ac247acc31363261 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:41.934307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:41.934395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.032713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a43859-c7c5-4341-9d18-c071fee1637f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.035981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c36a94-4bc0-42fe-9e6d-62f5d64abf95 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.067855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.068489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.068772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.069049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.070155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.071649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.071757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.072323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.072609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.072888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.073286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.073561 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:42.073871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.074046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.075270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.083381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 10:59:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.084177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.093106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.098365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.176247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fa61b-4f9f-4c9a-b69a-43699cf5c62d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.183261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb752f-5072-4680-a0e5-56b700e24b48 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.209125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.209742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.210020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.210295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.211437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.212863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.214460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.218903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.221538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.222174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.222461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.222743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.223920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.225383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.227063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.227800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.299019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.304217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.317631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bfe1f-e2d7-4f8e-bf66-20dd4634b658 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.325486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56867f5-952e-4fa6-9eab-0de5cfbb1ed2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.352354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.352978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.353265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.353545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.354655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.356115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.357865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 39/78] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.358441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.360028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.361281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.361577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.361850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/a996e65d4f0b46038e59c86823161af1/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.363873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, user_id=a996e65d4f0b46038e59c86823161af1, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.365313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.437718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 40/79] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:42 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/a996e65d4f0b46038e59c86823161af1/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:42.438681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.463208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856159d6-4538-46e9-a652-2d3313ad841d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.501600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.502256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.502537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.502847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.504153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.506138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.512954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.512954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.533649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.562482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709e5a2-c9e2-494d-a373-ef6208f7a90a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.601708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.602605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.602921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.603205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] PATH_INFO: `/v3/projects/4e1c1064f3744c8c825b68757370fc1a` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.604382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] RBAC: Authorizing `identity:get_project(project_id=4e1c1064f3744c8c825b68757370fc1a)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.605854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.607730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/projects/4e1c1064f3744c8c825b68757370fc1a => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:42.608328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.621296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.633438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bb877-c49b-4d4c-8f6d-dee4d6098799 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.670421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.671122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.671403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.671682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/e3c50bbb7acb4f91ac247acc31363261/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.674213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, user_id=e3c50bbb7acb4f91ac247acc31363261, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.675750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.717197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795ba90-d279-46d7-9a2e-cd4220860836 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.746601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 41/82] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:42 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/e3c50bbb7acb4f91ac247acc31363261/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:42.748492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.756252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.756881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.757170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.757455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.758142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.760541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.844520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.850227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 42/83] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 10:59:42 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:42.851966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:42.904400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e97e38-4cb1-481c-a9be-fd705030bdea None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:42.986601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:42.987572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:42.988510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:42.989003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:42.990853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:42.991282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:42.992971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:42.999820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 42/84] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:42 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:43.007909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:43.049361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527f9832-5958-4ded-928a-f0ba7bd9c1bd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:43.091756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:43.092560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:43.092983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:43.093846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] PATH_INFO: `/v3/projects/4e1c1064f3744c8c825b68757370fc1a/users/a996e65d4f0b46038e59c86823161af1/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:43.096569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] RBAC: Authorizing `identity:create_grant(project_id=4e1c1064f3744c8c825b68757370fc1a, user_id=a996e65d4f0b46038e59c86823161af1, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:43.098783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd20477-c386-4469-ae91-1a06648abfeb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:43.121434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587dd29-2455-4ecc-9809-b092505f9240 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:43.162051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:43.162716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:43.163039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:43.163315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/a996e65d4f0b46038e59c86823161af1/roles/2af5dd46546147a59f096049e368abab` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:43.167041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, user_id=a996e65d4f0b46038e59c86823161af1, role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:43.168486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df025a-7098-4e49-85a8-310c5b7d42e0 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:43.174019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 43/85] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:42 2022] PUT /identity/v3/projects/4e1c1064f3744c8c825b68757370fc1a/users/a996e65d4f0b46038e59c86823161af1/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:43.264236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 43/86] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:43 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/a996e65d4f0b46038e59c86823161af1/roles/2af5dd46546147a59f096049e368abab => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:44.963178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4f31-ca8f-44b1-bbd5-72898ae4b41e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:44.963682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4f31-ca8f-44b1-bbd5-72898ae4b41e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:44.963788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4f31-ca8f-44b1-bbd5-72898ae4b41e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:44.965137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 44/87] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:44.970647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:44.971475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:44.971967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:44.979599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:45.000967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:45.000967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:45.081610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 44/88] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:45.619624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4f31-ca8f-44b1-bbd5-72898ae4b41e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:45.756164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4f31-ca8f-44b1-bbd5-72898ae4b41e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:45.780509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:45.792998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:45.793963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:45.794440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:45.794953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:45.806680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-9c5a4caa-9228-4059-88d7-6d0643e352dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:45.811752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:45.812601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:45.813352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:45.816374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 45/89] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:45.822443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:45.856610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25791-fa6e-4fd9-933b-ddddac360bfe None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:45.857171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25791-fa6e-4fd9-933b-ddddac360bfe None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:45.857626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f25791-fa6e-4fd9-933b-ddddac360bfe None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:45.859202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 46/90] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:45.864743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac120af-1269-448b-b7e8-f39661185995 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:45.869929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac120af-1269-448b-b7e8-f39661185995 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:45.872214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac120af-1269-448b-b7e8-f39661185995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:45.911865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0ac120af-1269-448b-b7e8-f39661185995 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:45.913099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0ac120af-1269-448b-b7e8-f39661185995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:46.040645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 47/91] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:46.048288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca89508-64d9-4553-ae7f-14ef9f1a2495 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.104929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.105543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.105821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.106094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.106768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:46.107146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:46.108521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:46.111388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96add-0818-440d-ba16-eace044dc420 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.111680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96add-0818-440d-ba16-eace044dc420 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.111957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f96add-0818-440d-ba16-eace044dc420 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.113536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 48/92] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:46.116218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.119196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.119491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.119831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.120497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 45/93] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:46.126499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:46.147703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:46.149134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:46.229706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 49/94] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:46.311026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f75ba-f716-46f3-af9d-4be587d9ab40 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.352895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.353524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.353803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.354076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.359688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:46.361233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:46.363391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 46/95] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:46.368971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:46.437359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.492939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:46.510254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64a44c-82c1-41fc-838e-8ca2e97c66ce None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.548538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.549170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.549504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.549785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.614666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.635809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:46.637553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:46.654614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641b9f5-f118-40f2-bd75-ba2c2c494fb6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.690453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.699967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.705659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.705659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.705659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.714694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-24d857a1-180f-4cf8-9e7b-b7543c5140a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:46.721224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:46.722141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:46.722947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 47/96] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:46.723586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fca62-a0ee-4c9c-a9ee-bceac052192a None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.724147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fca62-a0ee-4c9c-a9ee-bceac052192a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.724147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fca62-a0ee-4c9c-a9ee-bceac052192a None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 48/97] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191aaa-510c-432d-85f1-bb22ce6cd608 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191aaa-510c-432d-85f1-bb22ce6cd608 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94191aaa-510c-432d-85f1-bb22ce6cd608 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 49/98] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:46.730452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191aaa-510c-432d-85f1-bb22ce6cd608 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:46.765519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 50/99] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:46.766290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:46.844506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94191aaa-510c-432d-85f1-bb22ce6cd608 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.866396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375357-f32b-4838-b616-c7466abb2595 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:46.868230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.878355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.879046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.879332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.879611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.890668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-14ec6142-1704-4e4a-97eb-18a11c58f8e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:46.895476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:46.896647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:46.897436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 50/100] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:46.898098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.898162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.898445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.903749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:46.904525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.904809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.905090 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.905795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:46.906102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:46.908040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a48572-8779-4dca-9629-2e400c30a42c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:46.920786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 51/101] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:46.920786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37454b12-9194-437c-aeba-bfa8622eb547 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:46.920786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37454b12-9194-437c-aeba-bfa8622eb547 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:46.920786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37454b12-9194-437c-aeba-bfa8622eb547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:46.921164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:46.921164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:46.926717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:46.936976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-37454b12-9194-437c-aeba-bfa8622eb547 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:46.939581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-37454b12-9194-437c-aeba-bfa8622eb547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:46.984974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 51/102] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:46.986450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 52/103] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804465ac-b440-4ac9-aba1-3a53f14aaadd None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804465ac-b440-4ac9-aba1-3a53f14aaadd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804465ac-b440-4ac9-aba1-3a53f14aaadd None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 53/104] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:47.015048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804465ac-b440-4ac9-aba1-3a53f14aaadd None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.127116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96afcc4-8ec7-4975-a8f7-e804f7819235 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.135935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804465ac-b440-4ac9-aba1-3a53f14aaadd None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.144860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.172157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.173883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.174166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.174881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.176706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.178619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.179176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.179984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.180396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.180835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.181673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:47.182098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.183666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2d7ee-fc48-4d79-919c-b7e02a4240f7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.185191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 52/105] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:46 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.185521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.186868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.194053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 54/106] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.195480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.198443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.198443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.216485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:47.218637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:47.305508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 55/107] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:47.305508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.312394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8d277-17f6-4960-a454-19dd96a2ce7a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.349891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.350722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.351174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.351761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] PATH_INFO: `/v3/users/e3c50bbb7acb4f91ac247acc31363261` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.352892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] RBAC: Authorizing `identity:get_user(user_id=e3c50bbb7acb4f91ac247acc31363261)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.354450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.356478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 53/108] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/users/e3c50bbb7acb4f91ac247acc31363261 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.362173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.401025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addf30a-4c52-425b-92ae-8d5182f8fd94 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.412990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.429909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.438921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.439541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.439818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.440090 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.440896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] RBAC: Adding query filter params (role.id=188e9c2030b54ebaadc92e64b3250166, scope.project.id=3f62f604498e452eb84940f7a90ee7b9, user.id=a996e65d4f0b46038e59c86823161af1) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:47.441203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.442651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.448462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 56/109] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/role_assignments?user.id=a996e65d4f0b46038e59c86823161af1&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.449237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.477103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d0ba4-2836-469b-97d6-44420d034a2c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.533007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.533953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.534384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.534861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.537035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.539391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.547250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 54/110] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.549731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.569684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb56f-5af0-4ae3-9af4-b3849c45ae4a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.607537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.608195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.608480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.608762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.621076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:47.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:47.627540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:47.628317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 57/111] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:47.629368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.675473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f292f-44ef-4021-af02-958e7e52312a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.689002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.695939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.712343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.712973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.713266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.713553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.714659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.716253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.717829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 55/112] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:47.718601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:47.740936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee2c8-91a2-4398-820c-a92cc3bd1d47 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.774102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.774715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.775026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.775338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.819389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4c2e0-e76d-4df8-9912-8c92f99db062 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:47.841431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.842936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.853427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:47.854030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:47.854318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:47.854596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:47.890512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.908379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:47.921375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:47.923429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:47.940008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 58/113] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:47.940644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.021525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 56/114] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:48.022372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.037562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc9cdf-fe49-418c-b288-ba5c215ae493 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.070326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.070983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.071269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.071546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.081932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:48.087280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:48.088327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:48.089295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 59/115] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:48.091397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.104175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.109045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.123334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed3054-751a-4d03-a2d5-6b62f8987e1d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.157744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.159903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.159903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.159903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.159903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:48.159903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.161200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.168945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 57/116] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.169733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.193913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098613a-2cb4-438b-87a7-0b21553e3fe3 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.228387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.228995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.229278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.229554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.230199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.231686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.267742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b11df-db6a-4f51-8d91-d364c523c050 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.300298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.300916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.301201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.301480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.302581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.304053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.305667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 58/117] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.306429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.319766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 60/118] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 10:59:48 2022] POST /identity/v3/projects => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:48.320607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.328172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.392492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.425783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38882f-4637-48ab-b6fe-af87a1d7fb0f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.442668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b42855-966e-45e2-a5d2-7ceed153ed40 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.463227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.463523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.463812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.464090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.465248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.466732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.468512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 59/119] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.469322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.492747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.493638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.494128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.494549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.495860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:48.496329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.498048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.509721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 61/120] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.510660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.565414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.583085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589793d8-105e-45b0-b70d-90e61946f88d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.618618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.619555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.619838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.620119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] PATH_INFO: `/v3/users/a996e65d4f0b46038e59c86823161af1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.621268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] RBAC: Authorizing `identity:get_user(user_id=a996e65d4f0b46038e59c86823161af1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.622844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.624226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df2de0-e604-4e21-bf9d-8074ad314fb9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.624476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 60/121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/users/a996e65d4f0b46038e59c86823161af1 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.625333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.652703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.665796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.666409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.666694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.667433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.668176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=3f62f604498e452eb84940f7a90ee7b9, user.id=e3c50bbb7acb4f91ac247acc31363261) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:48.668491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.670643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.687150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 62/122] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/role_assignments?user.id=e3c50bbb7acb4f91ac247acc31363261&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.687150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.736740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040dfd-7125-4830-8f53-565668639a0e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.784127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.784763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.785327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.785615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.786367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] RBAC: Adding query filter params (role.id=2af5dd46546147a59f096049e368abab, scope.project.id=3f62f604498e452eb84940f7a90ee7b9, user.id=a996e65d4f0b46038e59c86823161af1) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:48.787039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.789595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.800993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 61/123] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/role_assignments?user.id=a996e65d4f0b46038e59c86823161af1&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=2af5dd46546147a59f096049e368abab => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.802572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.828044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:48.830624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b02ade1-7171-48b0-915a-e93484b88aae None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.885387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.886317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.886604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.886912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] PATH_INFO: `/v3/users/2970d8323b8a47a687b9ab4e9490c164` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.905580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] RBAC: Authorizing `identity:get_user(user_id=2970d8323b8a47a687b9ab4e9490c164)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:48.908239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:48.911711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 63/124] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/users/2970d8323b8a47a687b9ab4e9490c164 => generated 289 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:48.916923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:48.933356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadce5b-3ea7-4a29-a791-37e65b5b68da None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:48.974265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:48.974957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:48.975248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:48.975531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:48.988789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:49.064487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:49.066006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:49.071241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03acb6-4a08-409a-8ce6-1647bfe488ae None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:49.089058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:49.121693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:49.122354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:49.122638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:49.123332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:49.124809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:49.126643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:49.128367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:49.133906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:49.194739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 62/126] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:49.195597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:49.250597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f5a6a-a200-4b72-a6b5-841e88476e0d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:49.268450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:49.291199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:49.291865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:49.292256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:49.292535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/2970d8323b8a47a687b9ab4e9490c164/roles/4e00bc50273145068c89533a0cde0b4f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:49.294845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, user_id=2970d8323b8a47a687b9ab4e9490c164, role_id=4e00bc50273145068c89533a0cde0b4f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:49.297017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32763-6364-41ff-a997-6646a29fe2f4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:49.298626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c8319-d304-4839-8e84-b6d2159e8b3f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:49.315466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:49.332576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:49.333227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:49.333510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:49.333786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] PATH_INFO: `/v3/projects/4e1c1064f3744c8c825b68757370fc1a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:49.335016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] RBAC: Authorizing `identity:get_project(project_id=4e1c1064f3744c8c825b68757370fc1a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:49.336527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:49.338250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 63/127] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:49 2022] GET /identity/v3/projects/4e1c1064f3744c8c825b68757370fc1a => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:49.344839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:49.370842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 65/128] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:49 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/users/2970d8323b8a47a687b9ab4e9490c164/roles/4e00bc50273145068c89533a0cde0b4f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:49.454296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02bf6d-7295-42da-a41c-7f6adf1f9c4d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:49.490683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:49.491337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:49.491623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:49.491899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:49.492653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] RBAC: Adding query filter params (role.id=188e9c2030b54ebaadc92e64b3250166, scope.project.id=4e1c1064f3744c8c825b68757370fc1a, user.id=a996e65d4f0b46038e59c86823161af1) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:49.492967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:49.494558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73316b72-056a-44bf-84bf-72e305b83396 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:49.501517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 64/129] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:49 2022] GET /identity/v3/role_assignments?user.id=a996e65d4f0b46038e59c86823161af1&scope.project.id=4e1c1064f3744c8c825b68757370fc1a&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:50.607809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bf19-fca5-43c9-8c3e-7bb091ce3863 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:50.608319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bf19-fca5-43c9-8c3e-7bb091ce3863 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:50.608404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bf19-fca5-43c9-8c3e-7bb091ce3863 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:50.609692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 66/130] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:50.615538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:50.615909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:50.616201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:50.634566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:50.635777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:50.677732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:50.699185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 65/131] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:50.973192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15bf19-fca5-43c9-8c3e-7bb091ce3863 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:51.069965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15bf19-fca5-43c9-8c3e-7bb091ce3863 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:51.102887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:51.103509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:51.103786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:51.104078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:51.105162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:51.106608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7ce55-bf2a-4754-abf0-63e455a94397 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:51.108192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 67/132] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:51.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:51.209178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da893ca4-136c-4814-bdd2-2a98f5b94784 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:51.242019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:51.242562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:51.242870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:51.243152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:51.243803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:51.245256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f2a747-f2ad-4488-a4f5-c0cf3c6b793c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:51.387806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 66/133] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 10:59:51 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:51.517722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17966-359f-4b90-a65f-c1e0a23778db None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:51.517920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17966-359f-4b90-a65f-c1e0a23778db None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:51.518193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17966-359f-4b90-a65f-c1e0a23778db None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:51.519608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 68/134] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:51.525670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:51.525959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:51.526238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:51.545043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:51.546306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:51.577080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:51.617267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 67/135] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:51.889000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b17966-359f-4b90-a65f-c1e0a23778db None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:51.934349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:51.996791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b17966-359f-4b90-a65f-c1e0a23778db None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.032145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.032773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.033054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.033328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.043615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:52.048932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:52.049770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:52.050583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 69/136] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:52.057391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:52.159834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f7e3b-d92e-4012-9eae-755b0e579287 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.194636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.195276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.195554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.195825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.196494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:52.196797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:52.198172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:52.206968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 68/137] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:52.212786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:52.318197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866aad8-c178-451e-b9d8-fe6bff082130 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.325615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:52.352375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.353016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.353295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.353569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] PATH_INFO: `/v3/users/2970d8323b8a47a687b9ab4e9490c164` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.354701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] RBAC: Authorizing `identity:get_user(user_id=2970d8323b8a47a687b9ab4e9490c164)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:52.356174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:52.357760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 70/138] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:52 2022] GET /identity/v3/users/2970d8323b8a47a687b9ab4e9490c164 => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:52.363963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:52.392829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:52.469781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f98c50-5f72-4608-bfc7-16856e957eaa None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.503332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.503944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.504223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.504498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.575172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:52.576663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:52.603402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:52.678563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 69/139] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 10:59:52.684060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:52.785102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e563b-acdd-4979-b023-5454865497e0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.819818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.820456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.820732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.821004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.822128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:52.826947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa8bcc-13ba-4123-b607-e38b04a91f62 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:52.826947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 71/140] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:52 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:52.832079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:52.930871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84a59f-4161-4ca8-bfb3-76c90285cffe None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:52.957701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:52.966334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:52.966971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:52.967257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:52.967531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:52.968318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] RBAC: Adding query filter params (role.id=4e00bc50273145068c89533a0cde0b4f, scope.project.id=3f62f604498e452eb84940f7a90ee7b9, user.id=2970d8323b8a47a687b9ab4e9490c164) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:52.978348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:52.978348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cceceb-6cc1-4222-bb8e-fc53454c6e52 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:52.978348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 70/141] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:52 2022] GET /identity/v3/role_assignments?user.id=2970d8323b8a47a687b9ab4e9490c164&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=4e00bc50273145068c89533a0cde0b4f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:53.745022 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3208d-536d-43d2-8200-b6e0c9a71929 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.745461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3208d-536d-43d2-8200-b6e0c9a71929 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.745533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f3208d-536d-43d2-8200-b6e0c9a71929 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.746927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 72/142] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:53.752837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad05ce-dcea-4d2a-867f-76b384100ea7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.753343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad05ce-dcea-4d2a-867f-76b384100ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.753412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad05ce-dcea-4d2a-867f-76b384100ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.772568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-daad05ce-dcea-4d2a-867f-76b384100ea7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:53.773759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-daad05ce-dcea-4d2a-867f-76b384100ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:53.844359 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 71/143] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:53.858519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a925-eecc-4ed4-bc66-1568d2ca4cd3 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.858803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a925-eecc-4ed4-bc66-1568d2ca4cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.859287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5a925-eecc-4ed4-bc66-1568d2ca4cd3 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.861198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 73/144] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:53.866345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.866638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.866942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.884574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:53.885736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:53.940394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bcf91-0d44-4ba9-a1d2-2a1ed6792bc7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.940770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bcf91-0d44-4ba9-a1d2-2a1ed6792bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.941274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bcf91-0d44-4ba9-a1d2-2a1ed6792bc7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.942629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 74/145] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:53.947624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:53.947911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:53.948188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:53.951941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 72/146] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:53.966351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:53.967579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:53.997280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.034640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 75/147] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:54.112421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.161961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.212744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119701-7a40-458b-a18d-c9d90f6f81f2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.218512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.253485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.254035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.254560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.254847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.265479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:54.271556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:54.272620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:54.273477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 73/148] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:54.279951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.343467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b8b23-8236-4ec2-a7f5-737d9a6a2fc7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.378775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.379505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.379788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.380058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.389882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:54.395224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:54.396412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:54.397299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 76/149] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 10:59:54.397961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.400178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92f49-3215-4f25-a6b7-bebb1a9d0d88 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.411120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.425980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.433162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.433762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.434039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.434311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.435006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:54.435313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.436708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.445456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 74/150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:54.445456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.504818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7952-69ec-465c-b064-9c9f3bdee845 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.543348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.544384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.544894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.545402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.546982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.548453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.550058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:54.551116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.561919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bc542-16b8-49f8-b5b4-ab0e017930f6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.599033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.599683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.599965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.600238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.600945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:54.601253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.602700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.610795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:54.611671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.649906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.653955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.659592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f7b7b-54e9-434b-869b-61096f21a6a0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.692169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.692788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.693072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.693342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] PATH_INFO: `/v3/users/23e3f18b5be045c0bd27eec1f2550244` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.705949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] RBAC: Authorizing `identity:get_user(user_id=23e3f18b5be045c0bd27eec1f2550244)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.707413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.709012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 78/153] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/users/23e3f18b5be045c0bd27eec1f2550244 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:54.711639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.716549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59541203-fec6-41c3-9ae0-b31ca5303710 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.751388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.751992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.752268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.752538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.753174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.754605 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.814095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31e417-6f78-489c-8b18-5fe919d73735 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.846229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:54.846871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:54.847155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:54.847495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] PATH_INFO: `/v3/users/23e3f18b5be045c0bd27eec1f2550244` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:54.848611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] RBAC: Authorizing `identity:get_user(user_id=23e3f18b5be045c0bd27eec1f2550244)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:54.850044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:54.851665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 79/154] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/users/23e3f18b5be045c0bd27eec1f2550244 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:54.851665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.860240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 76/155] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 10:59:54 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:54.861018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:54.887205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.964863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:54.975628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fcb33-78af-4c34-a0a0-e26df43418ba None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:54.979302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25b07-625a-46ac-baac-d816ea011439 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:55.014408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:55.015126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:55.015446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:55.016198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:55.017367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:55.018806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:55.018942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:55.019574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:55.019857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:55.020135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:55.021111 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 77/156] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:55.021749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:55.023326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:55.028099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 80/157] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:54 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:55.028238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:55.031939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:55.129806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:55.142483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48aaa5-3968-49d9-b64a-54c13c8ff67e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:55.154172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488411-ffbb-417b-84d6-8059881403c8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:55.181151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:55.181906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:55.182186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:55.182460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/users/23e3f18b5be045c0bd27eec1f2550244/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:55.185051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, user_id=23e3f18b5be045c0bd27eec1f2550244, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:55.186503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb87c69-abbc-4251-8db4-880a2965b4e3 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:55.192726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:55.193382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:55.193660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:55.193931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/users/23e3f18b5be045c0bd27eec1f2550244/roles/2af5dd46546147a59f096049e368abab` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:55.196084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, user_id=23e3f18b5be045c0bd27eec1f2550244, role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:55.197531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42e717-2039-43e0-b1c8-24f833db4c7b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:55.314382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 81/158] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:55 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/users/23e3f18b5be045c0bd27eec1f2550244/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:55.323396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 78/159] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:55 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/users/23e3f18b5be045c0bd27eec1f2550244/roles/2af5dd46546147a59f096049e368abab => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 10:59:57.179225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ad998-98b7-4107-8ec6-b62555741c66 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.179798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ad998-98b7-4107-8ec6-b62555741c66 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.179798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ad998-98b7-4107-8ec6-b62555741c66 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.181121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 82/160] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:57.186320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3030b4-1782-4355-a70f-f673378b59ac None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.186745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3030b4-1782-4355-a70f-f673378b59ac None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.186954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3030b4-1782-4355-a70f-f673378b59ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.206008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ee3030b4-1782-4355-a70f-f673378b59ac None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:57.207302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ee3030b4-1782-4355-a70f-f673378b59ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:57.220337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df5f79-5aec-4954-b30e-ea4deae6f73c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.220621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df5f79-5aec-4954-b30e-ea4deae6f73c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.220883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df5f79-5aec-4954-b30e-ea4deae6f73c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.222530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 83/161] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:57.224536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:57.229608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc9ab-3843-413f-a512-3aa48981de00 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.229885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc9ab-3843-413f-a512-3aa48981de00 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.230151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc9ab-3843-413f-a512-3aa48981de00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.251385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-726fc9ab-3843-413f-a512-3aa48981de00 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:57.254994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-726fc9ab-3843-413f-a512-3aa48981de00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:57.281224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 79/162] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:57.324946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 84/163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:57.492677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc839-d578-4689-a4e8-66714ed5866c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.492967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc839-d578-4689-a4e8-66714ed5866c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.493241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397dc839-d578-4689-a4e8-66714ed5866c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.494613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 80/164] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 10:59:57.500122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.500412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.500686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.512332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7999f0-f9a0-450e-bb53-32d917163b48 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.512601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7999f0-f9a0-450e-bb53-32d917163b48 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.512869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7999f0-f9a0-450e-bb53-32d917163b48 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.514444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 81/165] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 10:59:57.519562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:57.519773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.520277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.520550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.520991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:57.548413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 10:59:57.556432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 10:59:57.592258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 85/166] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 10:59:57.593111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:57.607004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:57.627340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 82/167] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 10:59:57.628174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:57.715995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:57.724201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ac510-ad86-481c-ac95-1d0b7f1b6379 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:57.746060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c12c55-03b5-4720-8deb-51e23bc1e397 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:57.850731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:57.851419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.852049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.852674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.855145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:57.855773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.856050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.856324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:57.857785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:57.859281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:57.863922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 83/168] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 10:59:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:57.863922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:57.869150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:57.874112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:57.875224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 10:59:57.876615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 86/169] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:57.880618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:57.952326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:57.963108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3270f1-2429-4566-b04a-16d21ad75664 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:57.983381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b8eb9-ad0c-46f7-b0c8-8b649e2551e0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:57.996429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:57.996986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:57.997258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:57.997526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.007454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:58.012204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:58.013079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:58.013891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 84/170] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 10:59:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:58.014571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.021522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.022135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.022416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.022690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.023370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.024805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.026661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.114346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f479c6-faa7-4c59-bf87-5406134dacf4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.148294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.148886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.149164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.149435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.150086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:58.150389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.151778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.159528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 85/171] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.167067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.167167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 87/172] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 10:59:57 2022] POST /identity/v3/users => generated 297 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 10:59:58.167167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.252093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.275938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1043a-038e-48a7-8f17-76c55a842cf8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.290014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545d216-e080-4ac4-a58e-b7f3d65edb88 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.310835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.320716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.321688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.322116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.325522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.325522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:58.325522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.325522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.333291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.333935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.334210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.334489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.335447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 88/173] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.336289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.352580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 10:59:58.357701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 10:59:58.361937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 86/174] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 10:59:58.362727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.459718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa55c5-619f-4421-90d3-56329076c4a6 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.488344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e70cbe-f73d-4e8a-b994-42c7be9de206 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.497164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.497805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.498083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.498787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] PATH_INFO: `/v3/users/efc27980189242e2a692933695a35ad3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.517277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] RBAC: Authorizing `identity:get_user(user_id=efc27980189242e2a692933695a35ad3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.519090 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.521354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 89/175] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/users/efc27980189242e2a692933695a35ad3 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.522238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.524956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.534119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.539366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.539366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.539366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] PATH_INFO: `/v3/users/23e3f18b5be045c0bd27eec1f2550244` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.539366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] RBAC: Authorizing `identity:get_user(user_id=23e3f18b5be045c0bd27eec1f2550244)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.539888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.541943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 87/176] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/users/23e3f18b5be045c0bd27eec1f2550244 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.542728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.584409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.653945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f690bd-371d-4dee-8e37-929294f12deb None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.669106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aca0ef-678d-428d-a525-629ee2021e1e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.698748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.699388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.699668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.699950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.701047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:58.701368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.703806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.709474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.710462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.711027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.711546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.712916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.718983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 90/177] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.718983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.720619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.722659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 88/178] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.723824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.793699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.838403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced1b4-84a2-405c-bff1-1690e7ec3864 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.842835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de2c4c-0363-4619-9b75-6fff91b884e6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:58.848392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:58.881911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.882649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.882969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.883397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] PATH_INFO: `/v3/users/23e3f18b5be045c0bd27eec1f2550244` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.884517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] RBAC: Authorizing `identity:get_user(user_id=23e3f18b5be045c0bd27eec1f2550244)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.884982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:58.885598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:58.885882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:58.885989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:58.886147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:58.887768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/users/23e3f18b5be045c0bd27eec1f2550244 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:58.888373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:58.970083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:58.971740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.008843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da4050-cf27-4235-9f7f-1b3656ce288a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.048175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.053384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.054042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.054334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.054638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/users/efc27980189242e2a692933695a35ad3/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.056802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, user_id=efc27980189242e2a692933695a35ad3, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.058244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.075983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 91/180] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:59.082128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:59.093696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.125046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 90/181] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 10:59:58 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/users/efc27980189242e2a692933695a35ad3/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 10:59:59.126171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:59.280104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52019207-89e6-4ba2-a2a0-62ad63edcf2e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.334133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fc5da-7497-4361-9c7b-bda3e3d19719 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.335109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.336158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.337497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.337966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.373172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.373890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.374243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.374651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.375842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.377564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.379663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 91/182] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:59 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:59.387266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:59.424136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.425779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.435877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.461297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.503797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876dd98-dee7-4b3f-860d-4d2cc200938c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.537188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 92/183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 10:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 10:59:59.543868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:59.573816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.574467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.574748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.575043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.575775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=18d19764be644b77b43468d3862b865e, user.id=23e3f18b5be045c0bd27eec1f2550244) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:59.576081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.577536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.585931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 92/184] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:59 2022] GET /identity/v3/role_assignments?user.id=23e3f18b5be045c0bd27eec1f2550244&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 10:59:59.667367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e9aef-0936-47d2-a0fb-621c593338e0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.679187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.704658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.705307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.705585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.705854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.707117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.708590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7748d72-192c-416b-aac1-d2b60ae252b6 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.710677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 93/185] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 10:59:59 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 10:59:59.716686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 10:59:59.852798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc069a3c-92bc-4215-aa2d-427d18c47ad4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 10:59:59.887478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 10:59:59.900384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 10:59:59.902981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 10:59:59.902981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 10:59:59.902981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 10:59:59.902981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] RBAC: Adding query filter params (role.id=2af5dd46546147a59f096049e368abab, scope.project.id=18d19764be644b77b43468d3862b865e, user.id=23e3f18b5be045c0bd27eec1f2550244) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 10:59:59.903498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 10:59:59.905289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754d95b-b627-4d57-b038-2824c556a979 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 10:59:59.912494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 93/186] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 10:59:59 2022] GET /identity/v3/role_assignments?user.id=23e3f18b5be045c0bd27eec1f2550244&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=2af5dd46546147a59f096049e368abab => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:00.591815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74dfc-7113-41d1-908d-c8e9dcbecbc0 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:00.592352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74dfc-7113-41d1-908d-c8e9dcbecbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:00.592352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d74dfc-7113-41d1-908d-c8e9dcbecbc0 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:00.594342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 94/187] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:00.599543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:00.599925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:00.600097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:00.619020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:00.620228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:00.646912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aa1eb-436c-4b14-b13b-1654f2b9bfd6 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:00.647203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aa1eb-436c-4b14-b13b-1654f2b9bfd6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:00.647477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971aa1eb-436c-4b14-b13b-1654f2b9bfd6 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:00.648817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 95/188] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:00.653656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:00.653938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:00.654210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:00.672718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:00.673906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:00.687901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 94/189] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:00.741373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 96/190] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:00.954742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.006444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.061072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae751e-f73b-4f46-9664-a43f3ecef742 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.080341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:01.097112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.097751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.098034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.098306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.108535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:01.113719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:01.114623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:01.115462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 95/191] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:01.121415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff98263-63af-4541-92b3-30f5990f108d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.122272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.129871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:01.157387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.158035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.158312 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.158636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.159773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.161212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.162988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 97/192] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:01.168081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.227677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66dbf6-aa0e-436e-8223-316108116c50 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.264415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.265585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.265968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.266251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.267035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:01.267353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.268708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.275977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c85c-66d4-4553-b642-8c3372bcf6e7 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.277783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 96/193] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:01.283242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.315346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.316001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.316239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.316511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.317159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.318593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613796f1-7f9c-439a-8aaa-556ff89a6575 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.326014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:01.385071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b409f-f898-4cbf-9fb2-7ae9c01f6b3a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.418331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.418993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.419274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.419546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] PATH_INFO: `/v3/users/80ec1b8a25ab46e28674ac19c8a71971` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.432989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] RBAC: Authorizing `identity:get_user(user_id=80ec1b8a25ab46e28674ac19c8a71971)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.434478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.436287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 97/194] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:01 2022] GET /identity/v3/users/80ec1b8a25ab46e28674ac19c8a71971 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:01.439444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 98/195] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:01 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:00:01.442407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.558001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:01.561673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ebcf7-7759-4395-84e7-467ea3629103 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.600698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.601375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.601635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.601883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.603138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.604635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.611005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 98/196] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:01 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:01.612885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fac06-1d51-4983-87be-107776ecd5c7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.613308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:01.613383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fac06-1d51-4983-87be-107776ecd5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.613550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fac06-1d51-4983-87be-107776ecd5c7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.615154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 99/197] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:01.620815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.621100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.621375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.641679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:01.642936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:01.671630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:01.725615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af024184-da37-4321-92e8-d4aa6f7decb6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:01.726320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 100/198] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:01.763527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:01.764183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:01.764461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:01.764739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/users/80ec1b8a25ab46e28674ac19c8a71971/roles/4e00bc50273145068c89533a0cde0b4f` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:01.767214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, user_id=80ec1b8a25ab46e28674ac19c8a71971, role_id=4e00bc50273145068c89533a0cde0b4f)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:01.768647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:01.858609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 99/199] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:01 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/users/80ec1b8a25ab46e28674ac19c8a71971/roles/4e00bc50273145068c89533a0cde0b4f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:00:01.996814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:02.104974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bcbd2-737d-4882-8649-b4c7ba174010 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:02.140937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:02.141916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:02.142394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:02.142913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:02.154071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:02.159291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:02.160360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:02.161342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:02.163293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 101/200] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:02.172466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:02.203072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:02.277717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c355f7a-3c8e-4bdf-a16c-383c2c8af497 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:02.313779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:02.314393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:02.314674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:02.315054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:02.315729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:02.316034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:02.317413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:02.326319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 100/201] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:02.332480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:02.356426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:02.437825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae486d37-6b51-43d2-bc96-2f81179da405 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:02.471702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:02.472331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:02.472607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:02.472881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] PATH_INFO: `/v3/users/efc27980189242e2a692933695a35ad3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:02.474016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] RBAC: Authorizing `identity:get_user(user_id=efc27980189242e2a692933695a35ad3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:02.475496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:02.477248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 102/202] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:02 2022] GET /identity/v3/users/efc27980189242e2a692933695a35ad3 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:02.484432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:02.592053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:02.605847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398e2b2-859f-4e39-9fcb-9f2d2190ac6b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:02.642261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:02.642896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:02.643181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:02.643460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:02.711904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:02.713295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:02.804180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:02.816773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 101/203] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:02.822362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:02.909943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:02.922413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07162e7d-c401-479f-8e43-11968f7d47ef None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:02.956587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:02.957217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:02.957497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:02.957771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:02.958985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:02.960416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae78411-744b-4111-b483-740d504247e9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:02.962119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 103/204] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:02 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:02.967998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:03.072443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1c6c-300c-495f-b229-3bcbb416ca2d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:03.110006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:03.110616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:03.110924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:03.111200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:03.112029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] RBAC: Adding query filter params (role.id=188e9c2030b54ebaadc92e64b3250166, scope.project.id=18d19764be644b77b43468d3862b865e, user.id=efc27980189242e2a692933695a35ad3) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:03.112339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:03.113960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e273d-b9db-4408-a33c-3e9bc1183f83 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:03.120207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 102/205] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:02 2022] GET /identity/v3/role_assignments?user.id=efc27980189242e2a692933695a35ad3&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:03.835151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2c76-45e6-4079-a796-8509cc3b1253 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:03.835681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2c76-45e6-4079-a796-8509cc3b1253 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:03.835681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2c76-45e6-4079-a796-8509cc3b1253 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:03.837070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 104/206] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:03.842279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa13-6536-4364-9a9c-130acfd51e19 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:03.842648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa13-6536-4364-9a9c-130acfd51e19 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:03.842896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa13-6536-4364-9a9c-130acfd51e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:03.861171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cdcfaa13-6536-4364-9a9c-130acfd51e19 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:03.862355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cdcfaa13-6536-4364-9a9c-130acfd51e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:03.909593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:03.932411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 103/207] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:03.936247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429061e-01af-4340-a2d2-dcc08b03c3d1 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:03.937127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429061e-01af-4340-a2d2-dcc08b03c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:03.937362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429061e-01af-4340-a2d2-dcc08b03c3d1 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:03.938759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 105/208] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:03.944581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71a08-a127-4977-a383-a4197d99b2d6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:03.944857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71a08-a127-4977-a383-a4197d99b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:03.945134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c71a08-a127-4977-a383-a4197d99b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:03.962957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d6c71a08-a127-4977-a383-a4197d99b2d6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:03.964124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d6c71a08-a127-4977-a383-a4197d99b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:04.031134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 104/209] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:04.032099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2625-d84d-44ea-9cf4-15555fe2871d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.032381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2625-d84d-44ea-9cf4-15555fe2871d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.032650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa2625-d84d-44ea-9cf4-15555fe2871d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.034218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 106/210] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:04.039218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.039506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.039783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.057660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:04.058848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:04.125704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 105/211] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:04.202778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa2625-d84d-44ea-9cf4-15555fe2871d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.298928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.305669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa2625-d84d-44ea-9cf4-15555fe2871d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.342183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.343361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.343597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.343707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.343893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.354230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:04.359600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:04.360602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:04.361537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 107/212] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:04.368271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.389708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.404382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fc0fb-056a-46dd-8050-8832f668d37f None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.438557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.439216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.439492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.439759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.440834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.442269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.444010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 106/213] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:04.444604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.471306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a508b83-e1c6-4adb-b408-9cd80df333af None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.504259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.504851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.505122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.505388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.506065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:04.506359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.507747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.516153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 108/214] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:04.516904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.546563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22daac05-67be-46c3-84a5-d321fba028e6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.557031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.580178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.580797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.581075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.581350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.591590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:04.593604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:04.597340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:04.598427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 107/215] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:04.599233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.618046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a67c9-7ec0-4b6e-9f2b-27c319b86f5a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.651301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.651906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.652186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.652465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.653115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.654538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.695223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342130a-8b9c-4210-a57a-adcfe123133e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.728494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.729135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.729416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.729689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] PATH_INFO: `/v3/users/313b23a0783f492d97fd749d9af31ff5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.742711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] RBAC: Authorizing `identity:get_user(user_id=313b23a0783f492d97fd749d9af31ff5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.744251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.745936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 108/216] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/users/313b23a0783f492d97fd749d9af31ff5 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:04.746707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.763925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 109/217] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:04 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:04.765140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:04.792410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.875729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ac823-844c-4b18-82f6-fb460e9440b7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.879223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13741-7011-4b78-a913-79779e6a266a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:04.912593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.913200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.913495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.913755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.914491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:04.914929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.916330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.917259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:04.917888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:04.918188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:04.918466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:04.919241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] RBAC: Adding query filter params (role.id=188e9c2030b54ebaadc92e64b3250166, user.id=313b23a0783f492d97fd749d9af31ff5, scope.system=all) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:04.919554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:04.921063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:04.922912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:04.925983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 109/218] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:04.928752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 110/219] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/role_assignments?user.id=313b23a0783f492d97fd749d9af31ff5&scope.system=all&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:04.931583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:05.043545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:05.053637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af42b77-ee0f-4a6b-be25-b269c2de2ac4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:05.095404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:05.096053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:05.096336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:05.096703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] PATH_INFO: `/v3/users/80ec1b8a25ab46e28674ac19c8a71971` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:05.097920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] RBAC: Authorizing `identity:get_user(user_id=80ec1b8a25ab46e28674ac19c8a71971)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:05.099527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:05.101375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 110/220] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:04 2022] GET /identity/v3/users/80ec1b8a25ab46e28674ac19c8a71971 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:05.107235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:05.215031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1639a-2b2f-459d-bb62-d047bd901645 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:05.250366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:05.251299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:05.251786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:05.252236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:05.320730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:05.324399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:05.325852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:05.426886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 111/221] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 11:00:05.432400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:05.538667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2e7f2-72ce-4e84-b782-2e54fa6145e7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:05.575589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:05.576582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:05.577101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:05.577623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:05.578978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:05.580708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0eb05-efb1-43e2-8ea4-9eae3dc81fe2 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:05.582578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 111/222] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:05 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:05.588752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:05.688071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:05.691775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a7447-04e8-4662-946f-0aeb79da787b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:05.727772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:05.728379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:05.728659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:05.728928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:05.729655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] RBAC: Adding query filter params (role.id=4e00bc50273145068c89533a0cde0b4f, scope.project.id=18d19764be644b77b43468d3862b865e, user.id=80ec1b8a25ab46e28674ac19c8a71971) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:05.729959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:05.731440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb134a2-dcc8-4d14-81c4-f733d061e05b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:05.738344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 112/223] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:05 2022] GET /identity/v3/role_assignments?user.id=80ec1b8a25ab46e28674ac19c8a71971&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=4e00bc50273145068c89533a0cde0b4f => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:07.115201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3174-31d5-4ad3-8b34-a433f0ffe015 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.115703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3174-31d5-4ad3-8b34-a433f0ffe015 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.115799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c3174-31d5-4ad3-8b34-a433f0ffe015 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.117283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 112/224] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:07.122264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3438b6-d4fc-4d0b-b43f-fcbdae8e4d8e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.122635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3438b6-d4fc-4d0b-b43f-fcbdae8e4d8e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.122838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3438b6-d4fc-4d0b-b43f-fcbdae8e4d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.139902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-bc5b-43c6-aebd-4ffbe31d9f49 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.140188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-bc5b-43c6-aebd-4ffbe31d9f49 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.140462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50b0b-bc5b-43c6-aebd-4ffbe31d9f49 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.141294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3b3438b6-d4fc-4d0b-b43f-fcbdae8e4d8e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:07.141787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 113/225] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:07.142802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3b3438b6-d4fc-4d0b-b43f-fcbdae8e4d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:07.146767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751766-be63-4f95-8807-174861f77a2c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.147089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751766-be63-4f95-8807-174861f77a2c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.147364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba751766-be63-4f95-8807-174861f77a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.166130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ba751766-be63-4f95-8807-174861f77a2c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:07.167465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ba751766-be63-4f95-8807-174861f77a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:07.188626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:07.215612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 113/226] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:07.220275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575b17-cb94-4390-aef6-2a55d60dd1f0 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.220609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575b17-cb94-4390-aef6-2a55d60dd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.220882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575b17-cb94-4390-aef6-2a55d60dd1f0 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.222304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 114/227] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:07.227465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.227787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.228018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.235870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 114/228] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:07.246104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:07.247310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:07.313189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 115/229] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:07.486574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751766-be63-4f95-8807-174861f77a2c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.503131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.533075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:07.590807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba751766-be63-4f95-8807-174861f77a2c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.610727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493b6a8-8785-4209-9579-637a6b508488 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.624761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.625387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.625666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.625939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.635817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:07.641250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:07.642167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:07.642988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:07.643645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.648405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.649047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.649319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.649585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.660084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:07.662467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:07.664835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:07.665710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:07.666527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:07.667242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.747872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e9f98-0cdf-4a28-bcc5-736c51f32aba None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.748265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:07.766047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227a638-b493-49cf-98f5-60c179fe9698 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.781408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.782125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.782408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.782676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.783803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:07.785233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:07.786793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 116/232] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:07.787633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.804385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.804996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.805272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.805539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.806191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:07.806491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:07.807877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:07.816268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 117/233] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:07.817384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.868123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:07.884080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a540f-7f95-41a0-ba3b-57fd786b72f4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.914912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301071a-f395-4ceb-8f6c-90968f1f2f9d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:07.917116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.917715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.917992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.918262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.918941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:07.919246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:07.920631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:07.928396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 117/234] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:07.929174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:07.949906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:07.950507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:07.950862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:07.951141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] PATH_INFO: `/v3/groups` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:07.952335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:07.953760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:08.026165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dee414-c91b-4d17-94d2-788dc4c3987e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:08.034999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 118/235] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Oct 26 11:00:07 2022] POST /identity/v3/groups => generated 229 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:08.035860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:08.039288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:08.068412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:08.069066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:08.069341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:08.069615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] PATH_INFO: `/v3/users/313b23a0783f492d97fd749d9af31ff5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:08.070885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] RBAC: Authorizing `identity:get_user(user_id=313b23a0783f492d97fd749d9af31ff5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:08.072368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:08.079369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 118/236] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:07 2022] GET /identity/v3/users/313b23a0783f492d97fd749d9af31ff5 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:08.080490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:08.098931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:08.154469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9ca2-b8d1-4a94-9100-97b5d5c115e3 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:08.193700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b930b-e3b4-4af9-ac51-31ce8a6993a6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:08.200460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:08.201514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:08.201514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:08.201763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] PATH_INFO: `/v3/users/098d655f407e4308a767fbac54d2ba8b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:08.219385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] RBAC: Authorizing `identity:get_user(user_id=098d655f407e4308a767fbac54d2ba8b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:08.221446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:08.223847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 119/237] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:08 2022] GET /identity/v3/users/098d655f407e4308a767fbac54d2ba8b => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:08.229575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:08.237555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:08.238217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:08.239392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:08.239392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] PATH_INFO: `/v3/system/users/313b23a0783f492d97fd749d9af31ff5/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:08.241050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=313b23a0783f492d97fd749d9af31ff5, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:08.242848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3453955-b2d3-429d-9207-f239262b4a67 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:08.293180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:08.330946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70d106-d0fd-46f1-805a-db4d90aad935 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:08.333331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 119/238] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Oct 26 11:00:08 2022] PUT /identity/v3/system/users/313b23a0783f492d97fd749d9af31ff5/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:08.368568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:08.369508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:08.369933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:08.370351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:08.371302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] RBAC: Adding query filter params (role.id=4e00bc50273145068c89533a0cde0b4f, user.id=098d655f407e4308a767fbac54d2ba8b, scope.system=all) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:08.371760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:08.373534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe6379-61bc-4ade-be0d-25a0b4922639 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:08.380317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 120/239] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Oct 26 11:00:08 2022] GET /identity/v3/role_assignments?user.id=098d655f407e4308a767fbac54d2ba8b&scope.system=all&role.id=4e00bc50273145068c89533a0cde0b4f => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:10.196064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d5d8-e7a5-43ac-a4e4-f69dd6999f5b None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.196579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d5d8-e7a5-43ac-a4e4-f69dd6999f5b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.196688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d5d8-e7a5-43ac-a4e4-f69dd6999f5b None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.198122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 120/240] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:10.203160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8df3b-fb30-4e8b-a6fd-92aa9caf274d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.203566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8df3b-fb30-4e8b-a6fd-92aa9caf274d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.203771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8df3b-fb30-4e8b-a6fd-92aa9caf274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.222580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-96e8df3b-fb30-4e8b-a6fd-92aa9caf274d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:10.223924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-96e8df3b-fb30-4e8b-a6fd-92aa9caf274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:10.291995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 121/241] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:10.505778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1ea21-780f-4940-8bba-9f3a9d4d6de4 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.506104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1ea21-780f-4940-8bba-9f3a9d4d6de4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.506291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1ea21-780f-4940-8bba-9f3a9d4d6de4 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.507776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 121/242] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:10.512766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.513061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.513342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.520054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f831a5-b81e-4610-8aec-5fbf1c0085ea None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.520340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f831a5-b81e-4610-8aec-5fbf1c0085ea None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.520623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f831a5-b81e-4610-8aec-5fbf1c0085ea None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.522030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 122/243] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:10.526785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.527108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.527390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.533645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:10.536058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:10.545532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:10.546698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:10.590962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:10.607074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 122/244] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:10.613683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 123/245] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:10.614133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:10.872634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c82b2f-628c-4ab9-bc60-9924d2751ead None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:10.922402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:10.922402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:10.922632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:10.922632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:10.922632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:10.922632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:10.922947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:10.922947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:10.922947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 123/246] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:10.922947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:10.968977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:10.985058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48504f-90f0-482f-881e-822cdf8d21d3 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.012706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb426ee0-7ad7-490a-86c2-3b77f02b948b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.020232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.020860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.021141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.021414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.031417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:11.036198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:11.037119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:11.037963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 124/247] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:11.038679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.050757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.051409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.051683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.051950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.062298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:11.067097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:11.068038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:11.068817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 124/248] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:11.069480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.107792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:11.140761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab32e3-f472-4974-9c75-8505734e6318 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.166282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c3d00-963b-414d-90f2-55f2a9729453 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.173688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.174283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.174561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.174859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] PATH_INFO: `/v3/groups` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.175478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.176926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.180136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:11.200096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.200691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.200969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.201242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.201921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:11.202225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.203637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.211794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 125/249] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:11.217075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.277204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 125/250] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Oct 26 11:00:11 2022] POST /identity/v3/groups => generated 226 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:11.283175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.314067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60138ad9-9853-4878-af8f-47baeaf03d26 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.354153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.354778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.355092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.355351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.356028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:11.356333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.358497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.369121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 126/251] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:11.375243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.396233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454667ae-313a-44ac-8cc6-9bce10f918e7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.423444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:11.426200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:11.433974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.438106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.438106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.438106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] PATH_INFO: `/v3/users/313b23a0783f492d97fd749d9af31ff5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.438106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=313b23a0783f492d97fd749d9af31ff5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.438650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.440324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 126/252] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:11 2022] GET /identity/v3/users/313b23a0783f492d97fd749d9af31ff5 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:11.446211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.491936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7270-6a75-4be8-8fb1-7b440e4bf22b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.531682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.533899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.533899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.533899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] PATH_INFO: `/v3/users/098d655f407e4308a767fbac54d2ba8b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.534153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] RBAC: Authorizing `identity:get_user(user_id=098d655f407e4308a767fbac54d2ba8b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.535568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.537366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 127/253] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:11 2022] GET /identity/v3/users/098d655f407e4308a767fbac54d2ba8b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:11.545720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:11.565968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ec52-e492-43a0-85a4-2ade85f5bfd6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.608244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.609227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.609717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.609996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.612854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] RBAC: Adding query filter params (role.id=188e9c2030b54ebaadc92e64b3250166, user.id=313b23a0783f492d97fd749d9af31ff5, scope.system=all) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:11.612854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.612854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d34273-6190-4b09-9f45-7919a481a835 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.626348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 127/254] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Oct 26 11:00:11 2022] GET /identity/v3/role_assignments?user.id=313b23a0783f492d97fd749d9af31ff5&scope.system=all&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:11.668754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d2fb-1c7e-44a8-91b4-bc9455da93c1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:11.678203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:11.709259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:11.709941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:11.710305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:11.710593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] PATH_INFO: `/v3/system/users/098d655f407e4308a767fbac54d2ba8b/roles/4e00bc50273145068c89533a0cde0b4f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:11.712551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=098d655f407e4308a767fbac54d2ba8b, role_id=4e00bc50273145068c89533a0cde0b4f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:11.714529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e9d03-c41d-47e0-93e6-da4ef4b25f76 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:11.782436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 128/255] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Oct 26 11:00:11 2022] PUT /identity/v3/system/users/098d655f407e4308a767fbac54d2ba8b/roles/4e00bc50273145068c89533a0cde0b4f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:13.892908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ead0-c871-48bb-86ea-cb5ace1e3a1f None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:13.893434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ead0-c871-48bb-86ea-cb5ace1e3a1f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:13.893434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7ead0-c871-48bb-86ea-cb5ace1e3a1f None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:13.895390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 128/256] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:13.901841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecf6-923c-4a28-8e4f-36d748288981 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:13.902270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecf6-923c-4a28-8e4f-36d748288981 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:13.902465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93ecf6-923c-4a28-8e4f-36d748288981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:13.927870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ca93ecf6-923c-4a28-8e4f-36d748288981 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:13.929089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ca93ecf6-923c-4a28-8e4f-36d748288981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:13.949924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852b803-bacd-4b90-b96b-efe08afffc91 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:13.950512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852b803-bacd-4b90-b96b-efe08afffc91 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:13.950984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852b803-bacd-4b90-b96b-efe08afffc91 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:13.954129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 129/257] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:13.959148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421b7f-6d2a-4e99-a3f0-de04ca086640 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:13.959721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421b7f-6d2a-4e99-a3f0-de04ca086640 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:13.960213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421b7f-6d2a-4e99-a3f0-de04ca086640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:13.993568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e6421b7f-6d2a-4e99-a3f0-de04ca086640 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:13.995204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e6421b7f-6d2a-4e99-a3f0-de04ca086640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:14.021184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 129/258] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 26 11:00:14.085424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:14.097399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 130/259] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 11:00:14.164957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6f25-3830-4ce3-bc23-e54f18d6ec15 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.165589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6f25-3830-4ce3-bc23-e54f18d6ec15 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.166080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f6f25-3830-4ce3-bc23-e54f18d6ec15 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.167680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 130/260] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:14.173543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.173830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.174107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.200829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:14.202060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:14.214472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 131/261] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 11:00:14.396453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6f25-3830-4ce3-bc23-e54f18d6ec15 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.412439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f6f25-3830-4ce3-bc23-e54f18d6ec15 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.417042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.417899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.418389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.418870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.432021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:14.437566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:14.438461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:14.439305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:14.442932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 131/262] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:14.445141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.446453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.455381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e788f-5bfa-43c2-aee6-ee3ad730aa76 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.459567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9f7ba-9e69-4bbe-8b1f-c30714952852 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.462868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.463485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.463765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.464034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.464697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:14.465010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.466858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.466858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:00:14.466858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:00:14.467772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.471971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.472640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.472919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.473191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.490544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 132/263] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:14.495737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:14.501257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:14.503624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:14.505782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 132/264] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:14.507467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.510385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.522991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f13bf9b-d064-4507-b1ae-af1076934918 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.530304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb19f5a-836a-43ce-88ed-bb3d5a419ef0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.537313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.537959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.538240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.538512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.550698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.551761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.551761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:00:14.551761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:00:14.552993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.556037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.556037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.556037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.556037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.556683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:14.556968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.558965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.558965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:00:14.558965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:00:14.559865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.567803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 133/265] 10.222.0.42 () {56 vars in 1201 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:14.573931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.576599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 133/266] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:14.582400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.597415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2900ec4-6713-4484-8845-e1664a18ebfc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.602109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a924a6d-8af8-4ec6-9efc-4e137cd6cdf5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.604010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.605020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.605563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.606045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.608133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.614850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.614850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:00:14.614850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:00:14.616759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.618757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 134/267] 10.222.0.42 () {56 vars in 1201 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:14.620682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.621415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.621702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.621976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.624275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.624714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.626495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.633093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 134/268] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:14.634247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.638910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c051e87d-7271-474e-9ede-a2b7cf6242e2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.647360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.648241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.648789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.649276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.650647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.652970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 135/269] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:14.657116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e492e-20b6-4e16-b627-ce301ff43970 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.659417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.667777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.669112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.669354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.669629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/groups/83e425ecf4654477971c5cd9bde78e6b/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.674016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, group_id=83e425ecf4654477971c5cd9bde78e6b, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.675376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.675376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:00:14.675376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:00:14.677976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.680286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db65d1c-b961-4bd0-87eb-10dc72a3a96d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.684949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.686177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.687153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.687852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.700102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:14.705914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:14.706810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:14.707667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:14.710212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 136/270] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:14.711194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.723076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb5ad1-b7ae-4b6e-ad72-ce2b4ad7c5ff admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.727901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.728864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.729347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.729832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9/groups/83e425ecf4654477971c5cd9bde78e6b/roles/2af5dd46546147a59f096049e368abab` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.733069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f62f604498e452eb84940f7a90ee7b9, group_id=83e425ecf4654477971c5cd9bde78e6b, role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.734443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:00:14.734443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:00:14.734443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:00:14.737477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042cd7a-1db8-4cf6-a705-ec0ebbdf57bb admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.780453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 135/271] 10.222.0.42 () {58 vars in 1463 bytes} [Wed Oct 26 11:00:14 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/groups/83e425ecf4654477971c5cd9bde78e6b/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:00:14.784423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:14.848594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 137/272] 10.222.0.42 () {58 vars in 1463 bytes} [Wed Oct 26 11:00:14 2022] PUT /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9/groups/83e425ecf4654477971c5cd9bde78e6b/roles/2af5dd46546147a59f096049e368abab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:14.869842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b883e-4fba-4448-9b84-abba89fcc2cc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:14.908994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a30f7-e12d-4d6e-8879-6d7cf4cc90c3 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.909519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a30f7-e12d-4d6e-8879-6d7cf4cc90c3 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.909519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a30f7-e12d-4d6e-8879-6d7cf4cc90c3 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.912493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 138/273] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:14.924768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.925155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.925430 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.960128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:14.961523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:14.963360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:14.964150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:14.964319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:14.964617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:14.965334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:14.965643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:14.968036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:14.989188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 136/274] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:14.995052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.008683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecebd6e-90ba-43cd-b955-582ef981210b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:15.025145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:15.025789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:15.026072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:15.026349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:15.031699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:15.033690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:15.035628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 137/275] 10.222.0.42 () {56 vars in 1201 bytes} [Wed Oct 26 11:00:14 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:15.041297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.056330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:15.065477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3188f0-c6fd-4afa-be70-1b6bef7c8829 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:15.083637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:15.084291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:15.084603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:15.084905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:15.086140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:15.088289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:15.090387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 138/276] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 11:00:15 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:15.091290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 139/277] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:15.097013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.112364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51a79b-dd34-426c-9dc6-8f34867fed12 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:15.117439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:15.118398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:15.119094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:15.119596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/groups/83e425ecf4654477971c5cd9bde78e6b/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:15.121917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, group_id=83e425ecf4654477971c5cd9bde78e6b, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:15.124894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:15.215813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 139/278] 10.222.0.42 () {58 vars in 1463 bytes} [Wed Oct 26 11:00:15 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/groups/83e425ecf4654477971c5cd9bde78e6b/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:15.411001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.527786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa3f6b-5fda-4722-abe7-2d78f9ba1a01 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:15.564703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:15.565359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:15.565640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:15.565909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:15.581255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:15.586193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:15.589020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:15.589826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 140/279] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:15.593406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.665152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:15.751862 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e6f0d-70f9-4b3d-a4ea-0ec839437709 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:15.807061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:15.807664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:15.807950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:15.808229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:15.808953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:15.809271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:15.811259 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:15.820679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 140/280] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:15.829998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:15.910797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:15.978035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a451474-d024-47a4-b85a-d4b53fd4892c None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:16.018776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:16.019442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:16.019719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:16.019987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] PATH_INFO: `/v3/users/098d655f407e4308a767fbac54d2ba8b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:16.023490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] RBAC: Authorizing `identity:get_user(user_id=098d655f407e4308a767fbac54d2ba8b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:16.024969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973038d2-3cad-4e02-914f-db8e87eac837 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:16.026583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 141/281] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:00:15 2022] GET /identity/v3/users/098d655f407e4308a767fbac54d2ba8b => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:16.032849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:16.165307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:16.173114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2323e-b3e3-4984-a1e1-53619b5891f7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:16.214386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:16.215389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:16.215885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:16.216352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:16.217801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] RBAC: Adding query filter params (role.id=4e00bc50273145068c89533a0cde0b4f, user.id=098d655f407e4308a767fbac54d2ba8b, scope.system=all) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:16.218324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:16.220080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a24061-2366-424c-ab06-b456b9c69646 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:16.239367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 141/282] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Oct 26 11:00:16 2022] GET /identity/v3/role_assignments?user.id=098d655f407e4308a767fbac54d2ba8b&scope.system=all&role.id=4e00bc50273145068c89533a0cde0b4f => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:16.418932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db43e01-ec16-471a-899e-c80a3c78694e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:16.419224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db43e01-ec16-471a-899e-c80a3c78694e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:16.419419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db43e01-ec16-471a-899e-c80a3c78694e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:16.421595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 142/283] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:16.427231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:16.427511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:16.427795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:16.456146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:16.457396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:16.537517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 142/284] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 11:00:17.085438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db43e01-ec16-471a-899e-c80a3c78694e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.105078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db43e01-ec16-471a-899e-c80a3c78694e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.114850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.115760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.116261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.116750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.132256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:17.137606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:17.139142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:17.139959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:17.142545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 143/285] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 58 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:17.148286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.163902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e24cc-bb89-40e3-937b-e8eb029fbea2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.168782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.170167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.170649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.171163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.172046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:17.172550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.174346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.186626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 143/286] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:17.192089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.203145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1773bc-f59f-4429-91a0-ab498801944f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.210862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.211740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.212221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.212693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.214204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.216420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1087cb5-8195-4436-98da-1b4e1c18da1f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.218196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 144/287] 10.222.0.42 () {56 vars in 1201 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:17.223787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.232685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854f877-cac7-4a62-a0f3-c8cbc138e8cd None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.234436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854f877-cac7-4a62-a0f3-c8cbc138e8cd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.236650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a1f89-08ea-4e02-ab7b-498e6a477cb3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.237742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854f877-cac7-4a62-a0f3-c8cbc138e8cd None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.240058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 145/288] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:17.245948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.246947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.248302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.252219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.253091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.253572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.254062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.255757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.258023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.260394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 144/289] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:17.266012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.287695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ec4d7-6832-453f-958f-261501119a78 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.289541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:17.293849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:17.297485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.298508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.299046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.299524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e/groups/83e425ecf4654477971c5cd9bde78e6b/roles/2af5dd46546147a59f096049e368abab` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.301923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18d19764be644b77b43468d3862b865e, group_id=83e425ecf4654477971c5cd9bde78e6b, role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.304927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.307251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 146/290] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 26 11:00:17.389807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 145/291] 10.222.0.42 () {58 vars in 1463 bytes} [Wed Oct 26 11:00:17 2022] PUT /identity/v3/projects/18d19764be644b77b43468d3862b865e/groups/83e425ecf4654477971c5cd9bde78e6b/roles/2af5dd46546147a59f096049e368abab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:17.622555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.632859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b71319-5d12-4dd9-9df9-2e5316d58867 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.708730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.709726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.710265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.710747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.736955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:17.748366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:17.749287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:17.750107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:17.752821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 147/292] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 131 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:17.758338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.772636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a256bd-dad3-4eb4-93c7-dff03c200ed5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.783560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.784257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.784543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.784825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:17.785854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.787499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596753ca-165c-4625-92ea-9f31ed4ad86d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.797889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 146/293] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:17.803424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.811847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616da7a-abb5-40e3-a9f3-199a29433f25 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.812128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616da7a-abb5-40e3-a9f3-199a29433f25 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.812407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c616da7a-abb5-40e3-a9f3-199a29433f25 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.814740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 147/294] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:17.817885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5a33f-2ac3-45c5-a63f-6c85cac69bf9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.819974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.820258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.820537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.822591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.823293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.823571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.823903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] PATH_INFO: `/v3/groups/951c62f5e4324b5faff34bcb1f5358f2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.831240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] RBAC: Authorizing `identity:get_group(group_id=951c62f5e4324b5faff34bcb1f5358f2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.833234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.835355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 148/295] 10.222.0.42 () {56 vars in 1201 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/groups/951c62f5e4324b5faff34bcb1f5358f2 => generated 229 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:17.840672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.843534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:17.844719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:17.855449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb5066e-a5a7-487f-a8d0-454c0be93ceb admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.860847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.861537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.861820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.862095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] PATH_INFO: `/v3/projects/6d130249a55c4bbea044091c5d05b944` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.863315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d130249a55c4bbea044091c5d05b944)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.865259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cd30-1c43-4ba7-8f1c-20f1b6c66f68 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.867612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 149/296] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Oct 26 11:00:17 2022] GET /identity/v3/projects/6d130249a55c4bbea044091c5d05b944 => generated 344 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:17.868778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323d0d-c33d-44c2-b8c8-438ee2b3a948 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.869059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323d0d-c33d-44c2-b8c8-438ee2b3a948 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.869333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323d0d-c33d-44c2-b8c8-438ee2b3a948 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.870894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 150/297] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:17.879065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.879348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.879622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.898914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:17.900109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:17.911051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:17.918111 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:17.927597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 148/298] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:17.929343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:17.945110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788c65a-e0b7-43b2-9e7b-bbab4ff7b808 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:17.949977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:17.951330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:17.951686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:17.952023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] PATH_INFO: `/v3/projects/6d130249a55c4bbea044091c5d05b944/groups/951c62f5e4324b5faff34bcb1f5358f2/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:17.954098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d130249a55c4bbea044091c5d05b944, group_id=951c62f5e4324b5faff34bcb1f5358f2, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:17.957320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:17.991046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 151/299] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:18.023762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 149/300] 10.222.0.42 () {58 vars in 1463 bytes} [Wed Oct 26 11:00:17 2022] PUT /identity/v3/projects/6d130249a55c4bbea044091c5d05b944/groups/951c62f5e4324b5faff34bcb1f5358f2/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:18.246588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.366857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc109057-ca17-428e-a6cd-37f415bb42e4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.402422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:18.403091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:18.403377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:18.403653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:18.414695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:18.419569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:18.420473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:18.421262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 152/301] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:18.424459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.431017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.436336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:18.556083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214a91f-f927-440b-8f90-2b838c6e6504 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.567701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0b51-7279-4d97-bd7e-bb9f883637ef admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.602408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:18.603065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:18.603350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:18.603685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:18.604362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:18.604922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:18.607711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:18.610770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:18.611433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:18.611714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:18.611988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:18.617395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 153/302] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:18.623675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.625513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:18.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:18.633753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:18.635337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:18.639295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 150/303] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:18.640584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.696607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:18.724165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:18.773637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8dfb2-c1b7-4990-a3f7-1c26b54e5dcc None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.777594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45447dd7-50d9-463b-ba99-75d44df160f2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.822770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:18.823376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:18.823653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:18.823923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:18.824907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:18.825772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:18.826250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:18.827000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:18.830039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:18.830568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:18.832192 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:18.845559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:18.850758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:18.922907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:18.925163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6850b37-02d6-4837-9558-d1221bf1052a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:18.976483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8089ea-7968-4100-9701-f2d4d7b68cd3 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:18.979866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.013657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.018687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.019465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.019837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.020234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.072414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 154/305] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 11:00:19.075208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b16e-28ff-449d-ac5b-8fe9034ab22e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.075463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b16e-28ff-449d-ac5b-8fe9034ab22e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.075799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738b16e-28ff-449d-ac5b-8fe9034ab22e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.077121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 155/306] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:19.077845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b16e-28ff-449d-ac5b-8fe9034ab22e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.117808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.119300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c3d31-6274-43c1-b492-1bcf196a1de7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.188697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738b16e-28ff-449d-ac5b-8fe9034ab22e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.237540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.238184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.238468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.238745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.239912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.241341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.242201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 152/307] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 11:00:19.243102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 156/308] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:19.246841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.247563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.248050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.253241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.259609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.276178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:19.279324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:19.294888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.367695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0a98e-df44-48da-9cc1-b259e17ed18b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.369745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 153/309] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:19.371604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.413650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.414383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.414735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.415110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.416326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.420735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 157/310] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:19.427060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.504209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39f9c6-6ce0-4a09-8bee-384aebfa84d6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.546063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.547087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.547578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.548569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] PATH_INFO: `/v3/projects/3f62f604498e452eb84940f7a90ee7b9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.550208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] RBAC: Authorizing `identity:get_project(project_id=3f62f604498e452eb84940f7a90ee7b9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.556960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.556960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 154/311] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/projects/3f62f604498e452eb84940f7a90ee7b9 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:19.560811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.564137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.601996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc7cd4-2976-4916-bd0a-767495c59300 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.603890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:19.683695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35afacc-5b05-4c44-bf83-799d7e0a6b28 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.695527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.696387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.696551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.696911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.697716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] RBAC: Adding query filter params (group.id=83e425ecf4654477971c5cd9bde78e6b, role.id=2af5dd46546147a59f096049e368abab, scope.project.id=3f62f604498e452eb84940f7a90ee7b9) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:19.698091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.699623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.712712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 158/312] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/role_assignments?group.id=83e425ecf4654477971c5cd9bde78e6b&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=2af5dd46546147a59f096049e368abab => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:19.726480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.728644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.729130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.729580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.731206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.732967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.734932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 155/313] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:19.741180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.859376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6740b7-71d4-4e7f-82e3-85f4baa9726c None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:19.898575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:19.899552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:19.900013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:19.900466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:19.901442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] RBAC: Adding query filter params (group.id=83e425ecf4654477971c5cd9bde78e6b, role.id=188e9c2030b54ebaadc92e64b3250166, scope.project.id=3f62f604498e452eb84940f7a90ee7b9) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:19.901940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:19.903822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:19.910410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:19.923418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 159/314] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/role_assignments?group.id=83e425ecf4654477971c5cd9bde78e6b&scope.project.id=3f62f604498e452eb84940f7a90ee7b9&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:20.012388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.056022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659ce57-c3ce-4a0a-8b27-038556db66c6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:20.097814 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:20.098453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.098728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.099029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.111704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:20.116818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:20.117747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:20.118559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 156/315] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:20.124842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:20.164914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.241492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26faeae-7dc1-4ab0-9826-a8f4a3c558db None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:20.285362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:20.285963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.286194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.286444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.287158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] RBAC: Adding query filter params (name=member) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:20.287461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:20.288871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c54fc-ee3d-4589-bf10-dbcecf8f6065 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:20.300314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 160/316] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:00:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:20.305924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:20.411793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65129-f346-4215-8dcc-4660271f6a02 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.412173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65129-f346-4215-8dcc-4660271f6a02 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.412439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c65129-f346-4215-8dcc-4660271f6a02 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.414042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 161/317] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:20.415577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21c107-fede-4ffa-b6a3-b4a6d26c2d71 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:20.421526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.421804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.422073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.441251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:20.442443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:20.452910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.458537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:20.459189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.459469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.459822 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.535858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 162/318] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:20.547682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:20.548589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:20.659431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 157/319] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:20.665742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:20.690971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.777096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e72874-5d79-4b6c-97f3-4e913d13edda None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:20.819872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:20.820487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:20.820747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:20.821019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:20.822253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:20.823784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:20.826004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 163/320] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:20 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:20.828024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:20.831501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:20.876909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.952865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:20.964086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93c8e-a4fd-47d6-bbf2-a27731044347 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:20.965149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d480bd-b5fa-437b-b5bb-718c2deb68b0 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.006704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.006996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.007376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.007595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.007695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.007874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.007969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.008151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.009016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.010456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b2411-e909-47d2-a28b-e7d86ab391fe None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.013386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 164/321] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Oct 26 11:00:20 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:21.014365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909e392-d3a8-4bfa-bb96-95ba7f1e2b4d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.014714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909e392-d3a8-4bfa-bb96-95ba7f1e2b4d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.015050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909e392-d3a8-4bfa-bb96-95ba7f1e2b4d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.016479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 165/322] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:21.019621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909e392-d3a8-4bfa-bb96-95ba7f1e2b4d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.024265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-6059a33c-12d5-4cf0-adbc-c065e88471ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:21.030243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:21.031196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:21.031988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 158/323] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:00:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:21.032702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.032775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.033337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.054403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:21.055771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:21.137141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 159/324] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:21.137716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909e392-d3a8-4bfa-bb96-95ba7f1e2b4d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.138587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.168644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.180571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.181186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.182889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.182889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.182889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] RBAC: Adding query filter params (group.id=83e425ecf4654477971c5cd9bde78e6b, role.id=188e9c2030b54ebaadc92e64b3250166, scope.project.id=18d19764be644b77b43468d3862b865e) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:21.182889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.184311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.192558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 166/325] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/role_assignments?group.id=83e425ecf4654477971c5cd9bde78e6b&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=188e9c2030b54ebaadc92e64b3250166 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:21.266860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e52178-fea4-46fd-96a0-ffd31dd103cf None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.312068 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.312696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.312978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.313257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.314037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:21.314347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.315784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.326189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 160/326] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:21.332401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.362850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.420091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.454093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea392f-3856-4fd7-991f-2a9b71430991 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.500460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.500460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.500765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.500931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.532555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.554777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdce46-84e9-4cf5-8bd2-1a625ad9f376 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.581089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.582569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.595486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.596124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.596408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.596687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.608609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:21.610548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:21.614340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:21.615352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:21.616151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 161/327] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:21.621255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.700120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 167/328] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:21.705577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.728329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e48f-e332-4f98-bb97-f329d8652bb8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.763781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.764381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.764662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.764933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.765590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:21.765893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.767289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.775607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 162/329] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:21.781253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.785080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.802315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f660dd5-5cfc-4c58-aa06-8f376319f74e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.825665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.836622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.837246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.837527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.837801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] PATH_INFO: `/v3/projects/18d19764be644b77b43468d3862b865e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.838931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] RBAC: Authorizing `identity:get_project(project_id=18d19764be644b77b43468d3862b865e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.840350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.841999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 168/330] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/projects/18d19764be644b77b43468d3862b865e => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:21.847641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:21.881783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c6581-c5b6-4e5f-a549-994ec88a34a9 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.913608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.914193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.914476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.914744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.944341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3ee51-b536-4e13-9918-ffdea9d0e772 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:21.975932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:21.976484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:21.976764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:21.977033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] PATH_INFO: `/v3/groups/83e425ecf4654477971c5cd9bde78e6b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:21.978110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] RBAC: Authorizing `identity:get_group(group_id=83e425ecf4654477971c5cd9bde78e6b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.979557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.980125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:21.981121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 169/331] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/groups/83e425ecf4654477971c5cd9bde78e6b => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:21.982171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:21.984879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:21.987348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:22.042473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:22.080674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 163/332] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 11:00:22.085423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610218-1321-4338-8b8f-ad298d7b1d42 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:22.086263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:22.116846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:22.117447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:22.117726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:22.117998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:22.118720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] RBAC: Adding query filter params (group.id=83e425ecf4654477971c5cd9bde78e6b, role.id=2af5dd46546147a59f096049e368abab, scope.project.id=18d19764be644b77b43468d3862b865e) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:22.119057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:22.120481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:22.126120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 170/333] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Oct 26 11:00:21 2022] GET /identity/v3/role_assignments?group.id=83e425ecf4654477971c5cd9bde78e6b&scope.project.id=18d19764be644b77b43468d3862b865e&role.id=2af5dd46546147a59f096049e368abab => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:22.188541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf33cd-765a-40d0-b0ab-b84359a6ed83 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:22.191057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:22.224869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:22.225503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:22.225780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:22.226108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] PATH_INFO: `/v3/projects/6d130249a55c4bbea044091c5d05b944` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:22.227222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d130249a55c4bbea044091c5d05b944)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:22.228662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:22.230334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 164/334] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:00:22 2022] GET /identity/v3/projects/6d130249a55c4bbea044091c5d05b944 => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:22.236921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:22.339491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633848a-3ec6-4451-bf12-59f1800c783f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:22.362459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:22.373597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:22.374231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:22.374516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:22.374793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] PATH_INFO: `/v3/groups/951c62f5e4324b5faff34bcb1f5358f2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:22.375932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] RBAC: Authorizing `identity:get_group(group_id=951c62f5e4324b5faff34bcb1f5358f2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:22.378686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c6877-6ecf-4227-bdaa-c6fb38cb7c99 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:22.379053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 171/335] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Oct 26 11:00:22 2022] GET /identity/v3/groups/951c62f5e4324b5faff34bcb1f5358f2 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:22.385025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:22.486626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78801c1-9452-4903-8983-83e18d5182d1 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:22.519767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:22.520359 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:22.520634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:22.520902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:22.521606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] RBAC: Adding query filter params (group.id=951c62f5e4324b5faff34bcb1f5358f2, role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=6d130249a55c4bbea044091c5d05b944) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:22.521911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:22.523358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383069-1077-4dba-bf2b-a947a86306d9 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:22.529480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 165/336] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Oct 26 11:00:22 2022] GET /identity/v3/role_assignments?group.id=951c62f5e4324b5faff34bcb1f5358f2&scope.project.id=6d130249a55c4bbea044091c5d05b944&role.id=27a106926ca14908b7e268013b8452a2 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:25.201900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3e198-2b98-4d1c-b2d0-171952f24120 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.201900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3e198-2b98-4d1c-b2d0-171952f24120 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.201900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3e198-2b98-4d1c-b2d0-171952f24120 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.202957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 172/337] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:25.208622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c1ea-f03c-4ed7-9dfc-26f07dd23ae3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.209521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c1ea-f03c-4ed7-9dfc-26f07dd23ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.209996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491c1ea-f03c-4ed7-9dfc-26f07dd23ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.236412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e491c1ea-f03c-4ed7-9dfc-26f07dd23ae3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:25.237944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e491c1ea-f03c-4ed7-9dfc-26f07dd23ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:25.282136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:25.331618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 166/338] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:25.604284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659dda4-2356-425d-b6d4-9adc195cb468 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.604887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659dda4-2356-425d-b6d4-9adc195cb468 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.605372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659dda4-2356-425d-b6d4-9adc195cb468 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.607733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 173/339] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:25.612744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8fe93-c1ac-4d06-b794-ee2ee8cdc882 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.613030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8fe93-c1ac-4d06-b794-ee2ee8cdc882 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.613364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8fe93-c1ac-4d06-b794-ee2ee8cdc882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.639301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-21c8fe93-c1ac-4d06-b794-ee2ee8cdc882 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:25.641013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-21c8fe93-c1ac-4d06-b794-ee2ee8cdc882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:25.696025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659dda4-2356-425d-b6d4-9adc195cb468 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:25.733208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 167/340] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:25.863700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659dda4-2356-425d-b6d4-9adc195cb468 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:25.869035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73072b08-310c-4ff9-8368-2ca8461bcabf None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.869583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73072b08-310c-4ff9-8368-2ca8461bcabf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.870078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73072b08-310c-4ff9-8368-2ca8461bcabf None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.871706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 168/341] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:25.878475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beb68-9c9a-42fe-a160-c44640d1899b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.879163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beb68-9c9a-42fe-a160-c44640d1899b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.880107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1beb68-9c9a-42fe-a160-c44640d1899b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.904648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cf1beb68-9c9a-42fe-a160-c44640d1899b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:25.906660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cf1beb68-9c9a-42fe-a160-c44640d1899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:25.919705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:25.920341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:25.920618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:25.920888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:25.938973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:25.940409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:25.960884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:25.961845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:25.962711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:25.962711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:25.962711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:25.967062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 174/342] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:25.971715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:25.992401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 169/343] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:26.040714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278453-9705-4ad5-8c7c-f3933823b355 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.041345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278453-9705-4ad5-8c7c-f3933823b355 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.041717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10278453-9705-4ad5-8c7c-f3933823b355 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.047648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 170/344] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:26.052317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.052842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.053317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.053421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.076920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:26.078466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:26.099858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.121718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753343ea-c943-4465-929f-5618bca1f628 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.147976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 171/345] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:26.149166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.151249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.151860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.152135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.152562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.153990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:26.154466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.156048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.169313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 175/346] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:26.179281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.310732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f807d1-1324-454f-8f7a-133f05a92184 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.351794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.352424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.352865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.353279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.353389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb076d6-d62c-4189-97f5-de9fadefc1e4 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.355260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.368046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.370328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.389528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.390345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.390787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.391228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:26.391458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:26.392784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:26.393709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:26.393709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.393709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:26.393709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 172/347] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:26.398675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.399512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.401099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.405804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.495276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff983f8-35ab-45cb-85c4-0f86867c2315 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.499033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 176/348] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:26 2022] POST /identity/v3/users => generated 246 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:26.500071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.545764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.548087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.548619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.549119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.566194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.568117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:26.590806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:26.592002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:26.592915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:26.592915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.592915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:26.595731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 173/349] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:26.597033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.639932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c5a5f-06c0-47f5-8edc-b792965766e2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.682111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.682936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.683293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.683637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.685078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:26.685463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.687680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.691746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.703037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 177/350] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:26.706354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.727626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.743962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd9f7c-393e-4847-8970-c49dd216ae51 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.781326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.781959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.782242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.782556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.793283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.794854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:26.810350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:26.811325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:26.812191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:26.812191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:26.812191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:26.812191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 174/351] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:26.812405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.824218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60b80f-8fa3-4d60-873f-a3bdd2369b3d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.861613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.862276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.862558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.862840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.863488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:26.863789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.865152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1481ee3-4319-457c-a877-7752c12628b0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 178/352] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff3e7-8d30-4fc8-b87c-ff65e7ded813 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff3e7-8d30-4fc8-b87c-ff65e7ded813 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ff3e7-8d30-4fc8-b87c-ff65e7ded813 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 179/353] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:26.881366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ff3e7-8d30-4fc8-b87c-ff65e7ded813 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:26.925625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a092e1a-37c7-46c2-a8fb-277d7686f560 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:26.939648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.960457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:26.961098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:26.961387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:26.961641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:26.962276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:26.963825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:26.967845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:26.993397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ff3e7-8d30-4fc8-b87c-ff65e7ded813 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:27.028187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:27.028793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.029077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.029396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.030039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:27.030348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:27.031961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a3e63-159e-4e41-aa2b-54822b277e98 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:27.036387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 175/354] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:26 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:27.040356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:27.060603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 180/355] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:27.067051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.067893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.068181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.089397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:27.091014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:27.158888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856217c3-6dd2-43f2-b2ea-f6f8fe53c122 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:27.186032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 181/356] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:27.187099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:27.226880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:27.227713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.228057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.228393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.229108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:27.230612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:27.279268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:27.315837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8b9be-9862-46c6-9892-d89993f7cb71 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:27.351050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:27.351957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.352425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.352866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.353732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:27.355965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:27.369450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 176/357] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:27 2022] POST /identity/v3/users => generated 248 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:27.484908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 182/358] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:27 2022] POST /identity/v3/users => generated 249 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:27.492671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:27.527187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:27.619444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949be3-6778-4f54-816e-b4bf161cd132 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:27.657889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:27.658507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.658781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.659080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.659759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:27.661154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:27.697255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.sql.core [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 11:00:27.697255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 26 11:00:27.697255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [parameters: {'id': '8dc2c82fda464ebda634ecf40ed46fff', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 26 11:00:27.697255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81744) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result.read() Oct 26 11:00:27.716606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return ref.to_dict() Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:00:27.717539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application yield resource Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise self.value Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application session.commit() Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 11:00:27.718466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.session.flush() Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application compat.raise_( Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 11:00:27.719340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:00:27.721330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result.read() Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:00:27.723276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [parameters: {'id': '8dc2c82fda464ebda634ecf40ed46fff', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:27.725242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 26 11:00:27.729896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:27.737897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 177/359] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Oct 26 11:00:27 2022] POST /identity/v3/roles => generated 144 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:27.738042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:27.847247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47688e-25e1-4d89-8308-6503fa9f6540 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:27.871198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:27.881727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:27.889057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:27.889057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:27.889057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:27.894502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-c7fcdaae-8237-4060-ba74-1bd0e49256d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:27.899334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:27.900238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:27.901036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 183/360] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:27.908966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:28.005707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:28.018158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041ae9d-f256-4e9d-906d-8786ad91e10a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:28.057670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:28.058283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:28.058562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:28.059390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:28.060062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:28.060360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:28.061825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafc97e-b158-4fe0-be93-0bda84503d99 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:28.070072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 178/361] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:28.823636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d1c9-109c-4072-9f6c-e7c9b9e35ecc None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:28.824646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d1c9-109c-4072-9f6c-e7c9b9e35ecc None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:28.825125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7d1c9-109c-4072-9f6c-e7c9b9e35ecc None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:28.826701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 184/362] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:28.831973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:28.832774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:28.833276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:28.852314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:28.853819 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:28.941144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 179/363] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:29.310500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7d1c9-109c-4072-9f6c-e7c9b9e35ecc None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:29.443544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7d1c9-109c-4072-9f6c-e7c9b9e35ecc None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:29.488940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:29.489585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:29.489861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:29.490129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:29.500966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-04c186ec-85ee-4b62-a2b2-01035d8639e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:29.510462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:29.511411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:29.512785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 185/364] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:29.518445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:29.689236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419b14-f5ed-4a87-96a0-9a41c9760b45 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:29.689937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419b14-f5ed-4a87-96a0-9a41c9760b45 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:29.690437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28419b14-f5ed-4a87-96a0-9a41c9760b45 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:29.692518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 186/365] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:29.697753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:29.698299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:29.698770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:29.713659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:29.721372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:29.722948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:29.748081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10436-3c88-428f-a226-e08baa621aaf None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:29.799666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:29.821747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:29.822363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:29.822644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:29.822941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:29.823631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:29.823936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:29.825906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:29.841052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 180/366] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:29.846928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:29.852012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 187/367] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:29.986371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3cedd-a5ef-48bb-aa0e-26a4daaba0a4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.021287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.021913 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.022189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.022459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.033365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.034884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.036655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:30.049904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:30.053256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:30.054116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:30.054116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:30.054116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:30.054116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 181/368] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:30.056794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.159048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.189116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7150895-1ec1-411d-865e-531a3f5987e5 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.237332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.238216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.238649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.239089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.240177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:30.240738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.242376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7088c9-0466-481e-b913-1d4aadfdd71e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.262303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 188/369] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:30.264586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7cd72-a67c-4331-b90b-ccc6f1e9650e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.265263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7cd72-a67c-4331-b90b-ccc6f1e9650e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.265694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7cd72-a67c-4331-b90b-ccc6f1e9650e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.268548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 189/370] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:30.269949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7cd72-a67c-4331-b90b-ccc6f1e9650e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.304375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df52e04-cd5d-48e1-983a-fa8d0b8c7170 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.312325 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.338392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.339326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.339829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.340732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.350959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-75ba1a81-44dc-4d8f-b7b2-8347748bdab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:30.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:30.357646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:30.358479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 182/371] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:30.359233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5e4d2-dbc2-4d91-9abb-6b17977e2499 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.359303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5e4d2-dbc2-4d91-9abb-6b17977e2499 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.360194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5e4d2-dbc2-4d91-9abb-6b17977e2499 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.360940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 183/372] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:30.361839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.366350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.366350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.380214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:30.381393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:30.389200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7cd72-a67c-4331-b90b-ccc6f1e9650e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.425755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.426376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.426655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.426959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.431313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.434251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.435728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.454148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 184/373] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:30.454148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-a525ce08-3f55-4ec2-a03e-ac3671deec6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:30.454385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:30.455262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 190/374] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.456177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.456945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.480517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:30.482404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:30.576523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 191/375] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:30.576523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.576778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6853ef8-9a84-4a6a-9334-489f22e65d99 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.593250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.626024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.626644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.628235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.628586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.630001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:30.630305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.631697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.646544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 185/376] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:30.655769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.726044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29405eba-2dfb-459d-b26f-1da535c0cece None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.741790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.790003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.790617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.790919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.791184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.791908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:30.792198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.793779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.827841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c020d-8169-4569-8a14-a5ac888be7dc None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.845391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 192/377] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:30.846355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.867458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.868128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.868513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.868855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.879591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:30.881054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:30.895897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:30.896839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:30.897695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:30.897695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:30.897695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:30.897695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 186/378] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:30.897915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:30.936328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:30.954085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe628a-6a05-469a-b6d3-6e87c691af5b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:30.987592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:30.988220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:30.988495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:30.988761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:30.998642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:31.001229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:31.003509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.004472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:31.005282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 193/379] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.006334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.013796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944a010-01e3-4bfe-b18f-89c20de1fd21 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.052377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.053349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.053802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.054277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.065766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.067261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:31.083023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.084083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:31.084947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:31.084947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.084947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:31.084947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 187/380] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.084947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.124756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44b0b-eca9-4227-b4d1-acbb2dfe9505 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.159646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.160268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.160542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.160812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.170806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:31.175513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.176436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:31.177252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 194/381] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.177917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.180369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.194999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66feb1b-a593-436d-9111-9b1cfcd6e205 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.228297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.228892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.229170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.229436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.230059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.230360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.231780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.248656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 188/382] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:31.249515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.261106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.278603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b412b-5fb5-472d-a8e0-d5e86f8aa9a4 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.311748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.312335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.312609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.312877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.313532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] RBAC: Adding query filter params (name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.313880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.315298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524787c1-c6d3-47bc-aba7-8db859c32811 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.322791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 195/383] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:31.324016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416571e-94f0-41b9-a15f-b5e34f970bbf None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.324854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416571e-94f0-41b9-a15f-b5e34f970bbf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.325127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416571e-94f0-41b9-a15f-b5e34f970bbf None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.326326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 196/384] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:31.327526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416571e-94f0-41b9-a15f-b5e34f970bbf None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.350014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070748c-4d7f-4057-ac8e-67b09a97cfd0 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.382949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.383632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.383910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.384181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.384807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.385109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.386480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.390055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.398277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 189/385] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:31.399159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.432557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416571e-94f0-41b9-a15f-b5e34f970bbf None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.465959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.466136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.466699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.467733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.468004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.468666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] RBAC: Adding query filter params (name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.468973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.470333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.478216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 197/386] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:31.479026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.497479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2bca8-2236-4789-b4ca-a1f559260235 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.530669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.531312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.531591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.531858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.539115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.540533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-fe95d5cb-b96e-4dc5-b633-b60602adaef3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:31.553264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.555174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 190/387] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.556272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.557170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.575110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:31.576298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:31.583064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504ca33-db02-4412-af87-c50e6df50603 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.608806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.616985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.617597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.617889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.623228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.628235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.629698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.640626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 191/388] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:31.641457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:31.644179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.645251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:31.646139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:31.646139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.646139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:31.646139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 198/389] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.646419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.713084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.758184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09932014-1fb2-4a95-ac5e-a62724e4efe3 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.765226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699d73-7cca-4fc9-bc4c-81038d9fd264 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.795125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.795958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.796237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.796502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.801153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.801988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.802446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.802811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.803240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.804663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.814862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.816442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:31.816682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.817632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:31.818562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 192/390] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.819137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:31.837743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:31.838843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:31.839973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:31.839973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:31.839973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:31.841963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 199/391] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:31.842970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:31.877638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.928831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd86a1-8ff8-4714-a2fe-0d3077b644e5 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.946717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6113061-59fa-40c4-a9fd-1daca3f4ebd1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:31.948360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:31.962221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.962834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.963120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.963392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.964099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.964405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.965868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.980087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:31.980708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:31.980986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:31.981260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:31.981894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:31.982198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:31.983581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:31.995498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 200/392] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.000461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.005836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 193/393] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.005836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.084624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.093740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5165938-22b6-46f7-83d7-fe7666780298 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.101598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1f655-9292-4a0d-9e34-343c0bc235ac None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.126348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.126978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.127258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.127530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.128272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.128519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.129971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.136064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.137240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.137519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.137789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.138419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.138721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.140179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.147857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 201/394] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.148652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.152998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 194/395] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.153942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.192628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.245458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa877c-8505-4abf-8b89-c4947439e5cd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.251166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b77fa-5010-4013-a308-a88cec336d24 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.277716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.278331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.278613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.278925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.283441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.284058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.284338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.284610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.289062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.290496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.292100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.293935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:32.304797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:32.305850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:32.306974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:32.306974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:32.306974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:32.307138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:32.307943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 11:00:32.309398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:32.309945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 202/396] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:32.309945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.310249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.311376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 195/397] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:32.312478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.404954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d892e-adb5-41ac-935d-27e39b576d07 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.407448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.411087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeebd74-8b1b-478e-8088-3efcb16f42d1 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.438621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.439179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.439491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.439720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.444219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.444863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.445139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.445412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.450093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.451594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.459281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=03aad2f885bc4dc183e934469676a42f, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.460695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:32.465283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:32.466151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:32.467029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:32.467029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:32.467029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:32.467029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 203/398] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:32.467256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.521810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.524932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 196/399] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:32 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:32.525754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.569810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b88a1-e320-454e-bb33-f7ad19c2c972 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.609802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.610503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.610787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.611125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.621346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.623002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:32.635785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:32.636822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 204/400] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:32.637722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.656712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ef877-a27a-4cb4-81ec-a77d523a8ed7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.693343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.693961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.694241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.694512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.695369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.696230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.696846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.709750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 197/401] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.710665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.722730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.759851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30697ba0-61ee-4517-b809-12aa7f3a0e90 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.775384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.796994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.797730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.798010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.798281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.799052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.799397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.800841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.824864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 205/402] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.824864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.827621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf635717-202a-43d7-a2ef-310887ea0c9a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.861996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.862600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.862908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.863182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.863823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.864128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.865546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.881718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 198/403] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.881718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:32.932921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47b507-bc4c-4bda-bfdc-21eda3711b8a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.956996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:32.972347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:32.972961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:32.973248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:32.973525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:32.974317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:32.974567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:32.976037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:32.990385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5b578-2c63-4d48-bff5-02d6b91597f6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:32.994240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 206/404] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:32.995170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.028826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.032474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.033083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.033361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.034253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.034253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.035714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.097448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d327a4-0e3e-4de0-b585-128bce1ffa70 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.130464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.131227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.131341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 199/405] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:32 2022] POST /identity/v3/users => generated 247 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:33.131433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.131690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.132559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.147321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.148538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:33.166970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:33.168139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:33.169176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:33.169176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:33.169176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:33.169176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 207/406] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:33.169451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.263846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b249c1-cf9c-431e-91df-785793548d43 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.291812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.296056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.308978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.309618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.309897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.310168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.320391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7285e3-8ee2-4156-a093-855f5f849517 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.321361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.325182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:33.337728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:33.338644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:33.339777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 200/407] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:33.344270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.379414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.380547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.381071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.381570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.395597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.397659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:33.419257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:33.420446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:33.421513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:33.421513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:33.421513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:33.423102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 208/408] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:33.424160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.476961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9af825-cfd8-49e4-9b37-1bc7c9f9b789 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.523295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.524124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.524554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.525012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.526369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:33.527159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.528540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.540694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5bd84-bf2d-4adf-a811-83e4d8c488b8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.549887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 201/409] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:33.551006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.580179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.580792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.581073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.581345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.582041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:33.582350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.583883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.585663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.598862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 209/410] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:33.598862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.616393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.682986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090612f-55bf-4bbd-b745-9790ecc2578d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.725690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd520-f012-4dc9-a1d7-3d9a63605cd0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.733666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.735102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.735611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.736108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.737400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:33.738336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.740230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.760510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 202/411] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:33.761660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.769052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.769786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.770139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.770487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.778504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.779979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:33.794626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:33.795543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:33.796381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 210/412] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:33.796868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:33.862217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.893767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d66fc3-b57d-41d7-b51c-572119c2d8a1 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.897622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:33.916057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3165e0-c6dd-40d1-8cdf-80f25b4c9d71 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:33.936943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.937835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.938282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.938694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/59ac752182c14071999766921f6b4f09/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.956733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:33.957501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:33.957899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:33.958242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:33.959006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=59ac752182c14071999766921f6b4f09, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.960459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.970509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:33.971964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:33.983670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:33.984922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 211/413] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:33.985781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:34.043306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 203/414] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:33 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/59ac752182c14071999766921f6b4f09/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:34.047965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:34.146385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feaa4d-3a50-4893-b7c2-3fdd066ca471 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:34.173100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:34.226441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:34.227355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.227636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.227912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.228641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:34.228948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:34.230476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:34.244582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a7b6-6bc6-418b-a03c-8b44095e9491 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:34.260372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 212/415] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:34.267193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:34.293523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:34.294107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.294506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.294849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.295982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:34.296291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:34.298503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:34.319437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 204/416] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:34.325423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:34.347320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:34.395882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ee6d-24d1-4413-8840-af8177c7d755 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:34.437971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:34.438637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.438938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.439214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/822597c72e114b549c4b51c2aeaf63d0/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.458726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=822597c72e114b549c4b51c2aeaf63d0, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:34.460210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694d30d-ef3e-4c31-b665-cb77fa675bf4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:34.507893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e785ac-c6dc-4107-b288-2da5fe6fefcc None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:34.544934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 213/417] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:34 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/822597c72e114b549c4b51c2aeaf63d0/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:00:34.549732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:34.550855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.551327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.551791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e3b32b5787164f1ba63e5e1e53f78174/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.570529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=e3b32b5787164f1ba63e5e1e53f78174, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:34.572454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395ac1f-c895-4b74-9df9-38acffb4a69c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:34.640471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 205/418] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:34 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e3b32b5787164f1ba63e5e1e53f78174/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:00:34.782387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f86-2bd2-4016-be42-d6c0709d250d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.783030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f86-2bd2-4016-be42-d6c0709d250d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.783569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f86-2bd2-4016-be42-d6c0709d250d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.785261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 214/419] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:34.791248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:34.791835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:34.792301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:34.814049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:34.815739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:34.887390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:34.896434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 206/420] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:35.301931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b9f86-2bd2-4016-be42-d6c0709d250d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:35.329682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:35.433712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b9f86-2bd2-4016-be42-d6c0709d250d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:35.480908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:35.481540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:35.481815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:35.482079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:35.498846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:35.503532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:35.504442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:35.505250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 215/421] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:35.512420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:35.649244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b313b9-dfad-4e28-bf4a-6a5c9e7ccdfb None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:35.704680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:35.705314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:35.705590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:35.705859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:35.706530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:35.706854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:35.708244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bc7d0-c9cd-4c00-91be-749d660a626f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:35.721186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 207/422] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:35.726919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:35.763955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5ee8-3277-4615-9e38-14c1ff691d36 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:35.764233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5ee8-3277-4615-9e38-14c1ff691d36 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:35.764507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d5ee8-3277-4615-9e38-14c1ff691d36 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:35.765835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 208/423] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:35.773864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:35.774144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:35.774419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:35.796175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:35.797347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:35.874854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:35.892538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c95c34-d8e2-4ff0-9bf3-0f05da7a2335 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:35.912268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:35.930484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:35.931164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:35.931441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:35.943134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:35.946580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 209/424] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 26 11:00:35.951005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:35.952586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:35.968102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:35.969039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:35.969918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:35.969918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:35.969918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:35.969918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 216/425] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:35.975015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.095973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148a61a-adc8-4003-8b7d-d10a7767be5c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.144000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.144615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.144897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.145173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.145816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:36.146123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:36.147521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:36.165646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 210/426] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:36.171192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.260958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.345199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:36.373955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72a0928-80df-4723-9ac4-7191aa87767e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.434925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.435474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.435754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.436026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.438546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618803-2362-49ce-825d-72426f5dfeeb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.443123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:36.452982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:36.454444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:36.470738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:36.471662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:36.472556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 217/427] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:36.480702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.505097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.505749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.506029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.506303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.528324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:36.533802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:36.535974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:36.537854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 211/428] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:36.547806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.655746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44554d46-191d-4aab-9eee-63b44d8ba825 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.688702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76fbad-4e96-4b61-aaa4-5f4cc49396da None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.696785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:36.700980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.701963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.702248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.702524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.703601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:36.708170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:36.708170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e6336-a396-4b0f-95de-97f5c2d8cc79 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:36.727206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.728175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.728648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.729131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.729394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 218/429] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:36.730306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:36.730689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c192-6c08-409a-acf1-bca72f2ac96f None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.731024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c192-6c08-409a-acf1-bca72f2ac96f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.731219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:36.731311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c192-6c08-409a-acf1-bca72f2ac96f None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.732488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 219/430] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:36.733063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cd77b-170e-46fe-9f8e-a2996d190932 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:36.738094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c192-6c08-409a-acf1-bca72f2ac96f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.747536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 212/431] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:36.753474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.753854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.754223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:36.773029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:36.774438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:36.807392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:36.853451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c192-6c08-409a-acf1-bca72f2ac96f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:36.862935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 213/432] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:36.863929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:36.914202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:36.914879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:36.915160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:36.915426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.038422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.039816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.049602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.056989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c47643-8577-429a-8a7c-1cd86f6f0b65 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.135573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.137167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.137651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.138131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.147170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.167794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.169612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.187455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 220/433] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:37.188424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:37.197856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:37.199444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:37.200314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:37.200314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.200314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:37.206680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 214/434] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:37.208003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.296223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4111c0-5cfa-43c2-a764-ffac29f09dce None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.309888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92664e-b5ac-4241-b400-26444f4f42f9 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.333848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.334469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.334742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.335089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.343282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.343902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.344183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.344528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.345229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:37.349976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:37.351060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:37.352112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:37.353458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 221/435] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:37.354256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.357067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.358515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.370238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-3afc2e58-c8d8-4e66-a9cf-8cad6047713e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:37.373222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:37.374123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:37.375012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:37.375012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.375012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:37.375012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 215/436] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:37.375970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d309475-622f-4bb7-8aaf-b8e89358181c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.376254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d309475-622f-4bb7-8aaf-b8e89358181c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.376524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d309475-622f-4bb7-8aaf-b8e89358181c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.377752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 216/437] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:37.378990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de97a-439e-49a6-8023-56d02544dfd1 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.379208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de97a-439e-49a6-8023-56d02544dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.379486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274de97a-439e-49a6-8023-56d02544dfd1 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.380729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 217/438] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:37.381428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de97a-439e-49a6-8023-56d02544dfd1 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.384369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.461204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d570ed-3161-45a7-9302-a1b43afd231d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.484209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274de97a-439e-49a6-8023-56d02544dfd1 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.495220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.496375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.496655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.496935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.497564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:37.497874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.499275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.510041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 222/439] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:37.510837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.521821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.522429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.522709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.523012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.523692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:37.523967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.525324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb261e1-6474-4154-80f4-82e9fcb59e77 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.533214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 218/440] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:37.534397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eac4b-a512-4bcc-8aa9-a2eda908a464 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.534680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eac4b-a512-4bcc-8aa9-a2eda908a464 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.534984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eac4b-a512-4bcc-8aa9-a2eda908a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.552477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-827eac4b-a512-4bcc-8aa9-a2eda908a464 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:37.553599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-827eac4b-a512-4bcc-8aa9-a2eda908a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:37.587807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.611905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc9f5c-b59a-4ed0-a31b-7d4f1a079f49 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.612048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.619631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 219/441] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:37.622435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.622732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.623023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.640704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:37.642121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:37.646371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.647013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.647297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.647573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.648217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:37.648527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.649905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.666081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 223/442] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:37.666081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.713826 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 220/443] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:37.714888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.786498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671c325-ac2a-4a6c-8cac-c1d5f07c29bd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.826875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.827625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.827966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.828309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.830498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.837303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.839723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:37.853388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:37.854501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:37.856248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:37.858007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1584a7-28bf-446f-adb0-7d3b6e2a6222 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:37.861606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 224/444] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:37.862465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.883257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:37.895086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:37.895868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:37.895965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:37.896233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:37.906515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:37.907987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:37.923408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:37.924495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:37.925522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:37.925522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:37.925522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:37.929800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 221/445] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:37.930585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:37.977358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ee01-8c19-448b-a0b3-84cf6336d5bf None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.012738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.013458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.013867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.014148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.021060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.022516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:38.034052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:38.035195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 225/446] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:38.036050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.042099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd021e-9606-49e3-839c-dfee8cb03a34 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.075375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.075972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.076276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.076550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.077328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.077581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.079029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.088665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.096170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 222/447] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.096844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.129778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.133049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b8aa6-2d73-4445-8861-88ff33c98459 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.166870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.168028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.168305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.168574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.178146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:38.184012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:38.184881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:38.185679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 226/448] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:38.186372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.196139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4ad73-7409-4e12-841d-e0292af49971 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.229578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.230681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.230982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.231253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.231873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.232172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.233517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.244413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 223/449] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.245216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.283459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5807a6-4f75-446f-826e-125f26465248 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.297106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.316865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.317505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.317772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.318045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.328353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:38.333556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:38.334633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 227/450] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:38.335475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.339478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.346456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d640911-b7ff-4036-89e7-257125d78b1f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.381014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.381606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.381885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.382159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.382868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.383181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.384624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.396774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 224/451] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.397565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.438925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852fde1-5743-4ade-8d56-c2b34bcb7201 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.470443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.471086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.471375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.471649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.482222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.483699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.492933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4159a8-5575-4e7c-8e5f-3c4e2f0dfc79 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:38.497899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:38.499000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:38.499893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:38.499893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:38.499893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:38.499893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 228/452] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:38.499893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.529495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.530096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.530382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.530487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.530656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.531410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.531661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.533028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.540916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 225/453] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.541611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.560118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.605748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f643f72-5239-45ac-baf4-942ccd2b5e79 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.639344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead1ce0-aed5-407e-b553-2bb85f790435 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.679451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.679615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.680042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.680139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.680317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.680409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.680581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.680678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.681223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.681518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.682933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.687496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.688909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.690833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.691698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:38.700554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:38.701484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:38.702377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.769507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.790338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310264c-ecbf-49e6-a0cc-de1ba45f4d19 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.790850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:38.806051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6c6f8-766f-44cc-b8cb-a2d67248bc4c None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.823712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.824309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.824593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.824864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.825578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=03aad2f885bc4dc183e934469676a42f) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.825887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.827358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.832970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 227/456] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/role_assignments?user.id=03aad2f885bc4dc183e934469676a42f&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.833846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.840083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:38.840699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:38.840976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:38.841242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:38.841892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:38.842193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:38.843629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:38.861619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 230/457] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:38.861619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:38.962048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26784861-54cc-4435-b3c2-d9b44ce02212 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:38.973881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb4e31-2b5f-49a1-bef2-aaf565705b78 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.004325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.004963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.005241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.005512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.016799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.017332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.017564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.017847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.018119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.018891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.020548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.030417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.031994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:39.034115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:39.035035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:39.035915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:39.035915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:39.035915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:39.035915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 228/458] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:39.036129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.042724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:39.050806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:39.051737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:39.052595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:39.052595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:39.052595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:39.052595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 231/459] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:39.052809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.143522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e690245-97ac-4f83-aecb-2255d71ee1f7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.167848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892053-ce44-4cf1-b9bc-6be0e6ff2b19 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.178173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.178773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.179082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.179348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.180060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:39.180471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.181987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.206005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.206197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 229/460] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:39.206306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.206389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.206459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.206459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.215069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.216932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:39.228775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:39.229644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:39.230518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 232/461] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:39.231599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.267586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.289156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.319918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658272d-de80-427a-8f8e-322b4281cc81 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.332686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992fe30-ff05-4b9c-9da8-d0c28d8f9c19 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.359364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.360233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.360512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.361028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.362070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:39.362378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.364261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.369939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.371102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.371385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.372102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.372770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:39.373083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.374943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.381363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 230/462] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:39.381363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.388108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 233/463] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:39.392840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.486029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea1dfb-cc2c-4091-8241-32ca613b35da None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.499124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.501619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5605fa-ab55-4706-89a7-cd4318be9186 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.520280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.521979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.522804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.523759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.524225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.541280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.541280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.541280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.541280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.541608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:39.541608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.542866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.554558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 234/464] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:39.555452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.596316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.597765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.705122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 231/465] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:39.705958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.714992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd9390-0645-429f-bdcb-97798ec7f494 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.727002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.754496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.755214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.755494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.755776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.765056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.766771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:39.779114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:39.780023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 235/466] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:39.780950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.823792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28d4b5-1a5c-4567-b499-0c7f27e7f2c5 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.824397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.860232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.860858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.861138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.861532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.868893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.870350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:39.882856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:39.883730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 232/467] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:39.884645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:39.891699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee78a-0017-4d01-a0b2-5986d7e981a1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:39.929365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:39.930023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:39.930305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:39.930580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/d256290285d44aafb8062ec2059b16ea/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:39.944552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=d256290285d44aafb8062ec2059b16ea, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:39.945987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:39.962680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:39.991693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5592-1cbf-4b37-b627-a900dd4f1d6a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.018585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 236/468] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:39 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/d256290285d44aafb8062ec2059b16ea/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:40.019514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.034311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.035018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.035305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.035579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.050312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.052208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:40.074311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:40.075334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:40.076186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:40.076186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:40.076186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:40.076186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 233/469] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:40.076885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.145180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.159060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da2a5-ffd5-41db-a5a9-98e74336779c None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.203578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.204569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.205069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.205530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.206455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:40.207784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.209420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.231874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f50f6-9157-4415-8ab3-702fde27a49c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.232056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 237/470] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:40.232056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.259065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.276868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.276868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.276868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.276868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.277210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:40.277210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.277210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.295412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 234/471] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:40.296344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.359082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c5db-9f8e-43bb-bebc-a350889123f0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.395461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.396109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.396429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.396708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.397341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:40.397645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.399146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.415435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 238/472] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:40.415435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.415973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83776c6c-8018-47b7-b03e-aabc37b1007d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.427211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.453612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.454539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.455042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.455514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.511396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.533633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.536116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.540766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343d06a-350e-4bbb-9ca0-3025f40921b8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.583827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.584482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.584771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.585052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.654375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 235/473] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:40.655298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.678420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.679908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.690727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.749722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.768542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6d8e9-9dc8-4b37-98c5-67b07a199426 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.809010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.809799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.810191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.810558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:40.818587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 239/474] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:40.818845 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:40.819845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.820580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:40.833455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:40.834531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:40.835644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:40.837034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 236/475] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:40.838000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:40.936179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39783e-a17f-4cfe-b42f-d16a30a4802e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.947691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:40.964285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281427b4-5fa3-40b3-adb7-98179d256b36 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:40.989344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:40.989994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:40.990277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:40.990554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.006529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.008639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.010430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.011298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.011582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.011858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.024078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.025525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:41.027094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:41.027975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:41.028802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:41.028802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:41.028802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:41.028972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.031301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 240/476] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:41.032421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:41.044824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:41.046100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:41.047022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:41.047022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:41.047022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:41.047022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 237/477] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:41.047287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.137715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b399aab-df01-4bee-99c8-834f82570180 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.171442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.172041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.172316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.172585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.173260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.173564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.174419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe2bfd-6016-4491-9a61-f760d7a12a95 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.175007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.191548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 241/478] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.191548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.211585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.212658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.213259 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.213541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.214403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.215446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.215446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.217187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a5d55-0181-476f-bfef-7041f3f3bac6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.229692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 238/479] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.230336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92848336-9739-4eef-8a3a-cfa8e9f66b31 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.230618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92848336-9739-4eef-8a3a-cfa8e9f66b31 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.230917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92848336-9739-4eef-8a3a-cfa8e9f66b31 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.235044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 239/480] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:41.235044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92848336-9739-4eef-8a3a-cfa8e9f66b31 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.294687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.298750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36972eeb-d8b2-4679-a759-84bb9928da9e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.335745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.336367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.336643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.336972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.337618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.337929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.339930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.341251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92848336-9739-4eef-8a3a-cfa8e9f66b31 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.355919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 242/481] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.355919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.379564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.380163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.380439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.380714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.381425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=59ac752182c14071999766921f6b4f09) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.381729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.383184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdb657-44c0-41b4-abfe-42451d32c1e6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.389544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 240/482] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/role_assignments?user.id=59ac752182c14071999766921f6b4f09&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.392354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.392640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.392942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.416332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:41.417279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:41.450238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.478072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8480eb-75b7-4d8b-bd82-587ebc5729db None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.507335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 241/483] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:41.508529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.519667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.520698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.521177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.521638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.531305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.533449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:41.547153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:41.548585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:41.549424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:41.552859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 243/484] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:41.560859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.616501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.671988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e4879-b507-48c7-8d6f-68ce4eff6b92 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.676838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6c002-5469-4629-ae7b-5a4bffadd067 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.707076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.707686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.707967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.708241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.708956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.709266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.710692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.717399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.717993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.719529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.720011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.720596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.728462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 244/485] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.729458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.735856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.735968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:41.744831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:41.746232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:41.747103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:41.749697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 242/486] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:41.755477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.847322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b5074-0a51-4ced-bcce-7c8f95b59c04 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.892115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951c2ec-ee35-4958-b97c-7920d07b13b3 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:41.896515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.897135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.897987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:41.898680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.898807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.899617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=822597c72e114b549c4b51c2aeaf63d0) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.900255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.902039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.909780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/role_assignments?user.id=822597c72e114b549c4b51c2aeaf63d0&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.911712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:41.949165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:41.949704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:41.949985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:41.950255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:41.950997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:41.951297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:41.952791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:41.967289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 243/488] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:41.977720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.048251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:42.126085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50e361-ca77-451c-971b-6bcd9675e6c1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.132404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc07f4-736a-4df9-a57b-e206a2a03021 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.176623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:42.177532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.177825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.178102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.182645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:42.183938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.184442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.184917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.186093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=e3b32b5787164f1ba63e5e1e53f78174) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:42.186647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:42.188391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:42.191643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:42.196054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 244/489] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/role_assignments?user.id=e3b32b5787164f1ba63e5e1e53f78174&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7a8e320f-30c0-4c60-9ea1-d6bb0c897514 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:42.201209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:42.202158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:42.203598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 246/490] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:42.208618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.247170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:42.325708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda49-fd79-4aad-b7c6-9d31fe36ca1e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.326053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda49-fd79-4aad-b7c6-9d31fe36ca1e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.326728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fda49-fd79-4aad-b7c6-9d31fe36ca1e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.328813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 247/491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:42.336002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ae556-0a5d-485f-9134-bff8ee339f06 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.336166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.336488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.336771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.368228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:42.369492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:42.415810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:42.416433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.416755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.417031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.417704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:42.418010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:42.419418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:42.429945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 245/492] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:42.438852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.493947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 248/493] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:42.564751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:42.571770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adccc1a-fda1-4c9a-b5fb-1187e695e18b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.609372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:42.610013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.610294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.610643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.625061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:42.626573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:42.642145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:42.643278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:42.644145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:42.644145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:42.644145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:42.644145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 246/494] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:42.649209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.731572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:42.795109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370c1b7-cb84-43ed-8615-16ced2734dbf None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.800632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.834556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:42.835433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:42.837007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:42.837423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:42.838208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:42.839497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:42.841091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:42.862295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 249/495] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:42.871713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:42.992534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477089e-6b07-443b-9296-a8eb0f4d98ff None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:42.993767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dfc22f-ed5b-4182-bdf0-797c49710b4d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.006372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.034955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.035984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.036459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.036925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.048051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.049882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:43.065863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:43.066951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:43.067873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:43.073617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.074599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.075125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.075652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.077183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 250/496] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:00:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:43.083705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.098234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:43.106654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:43.107590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:43.108511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:43.115217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 247/497] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:43.120847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.130898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.215571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ac94f-f3ee-42f4-a61c-25b2df88c104 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.253232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.253839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.254116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.254387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.255142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:43.255449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.256927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.274889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 251/498] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:43.283749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.293248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e116-346a-4fdc-aff9-3f0ec80eee04 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.327825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.328746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.329220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.329704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.330568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:43.331103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.332798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.336361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.343045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 248/499] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:43.355565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.412518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1055c8e-bdb0-408f-903c-7b41e105c2df None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.455797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.456436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.457903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.458188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.476384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.478666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:43.508084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:43.509042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:43.510935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:43.510935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:43.510935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:43.510935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 252/500] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:43.514041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.515750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.571626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453203e7-4ace-453e-b344-53c62c153ff7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.634199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.635233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.636220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.637567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.659287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.661049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.670941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:43.684811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:43.691603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:43.692520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:43.692520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:43.692520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:43.692520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 249/501] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:43.694697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.701165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fee40e-4393-43bb-bd64-758cb64e8d34 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.783614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.784166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.784454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.784742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.785401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:43.785712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.787230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.811622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 253/502] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:43.814548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.877673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c7089-bc7e-4611-bb12-6e9b20dcf7cc None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:43.914336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:43.942172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:43.942801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:43.943104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:43.943378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:43.944029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:43.944323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:43.945687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:43.974939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 250/503] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:43.975846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:43.984465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1cc11-f1cd-4f67-bd8a-da3bc36d70a9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.050168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.050756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.051070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.051346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.064344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.065843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:44.088062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:44.089086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:44.090051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:44.099488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 254/504] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:44.101625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.107118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.156601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3e914-b449-42d1-b1f4-7b09b2e529f1 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.196204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.196839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.197122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.197395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.207372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.209343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-b4f1dadc-51dc-4904-9aaf-2577fbec635f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:44.223329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:44.224502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:44.225494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:44.230808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 251/505] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:44.232162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c7ec-fad0-4df1-92c3-9fb74e216571 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.232448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c7ec-fad0-4df1-92c3-9fb74e216571 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.232721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0c7ec-fad0-4df1-92c3-9fb74e216571 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.238334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 252/506] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:44.239177 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0c7ec-fad0-4df1-92c3-9fb74e216571 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.268392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f980-93fd-4dc0-bd0e-031ab8f70dd7 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.292514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.312854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.313842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.314605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.315215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.316186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:44.316795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.318987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be7d71-533c-448c-aa2e-84f409bc6c20 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.334763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 255/507] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:44.336745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.337177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.338045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.360631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:44.361993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:44.369297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0c7ec-fad0-4df1-92c3-9fb74e216571 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.407005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.407951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.408398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.408825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.409709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:44.410184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.411832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.423115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.436105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 253/508] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:44.436105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.438475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 256/509] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:44.439695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.568150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.572310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734830cb-673b-46df-b9e6-d2ea78aedb57 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.609167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.609823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.610135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.610419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.613080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=03aad2f885bc4dc183e934469676a42f, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.614506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7491c-5ebf-43b0-9713-a8bf6fbae0bb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.642879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca41da5-0c1f-4fea-a370-74050abc3cfb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:44.679840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:44.680604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.680790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.681032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.692333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 254/510] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:44 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:44.707106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4d58-e655-43f7-820c-f093b7880332 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.707745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4d58-e655-43f7-820c-f093b7880332 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.708258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f4d58-e655-43f7-820c-f093b7880332 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.712063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 255/511] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:44.718621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.719757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.720271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.744423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:44.747181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:44.793329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.805778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:44.807974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f7762-7b1a-4dac-bfe4-6a37284eac4e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:44.841460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 256/512] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 26 11:00:44.842302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.949071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:44.951111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 257/513] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 26 11:00:44.953213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b03a-ba03-484b-86a8-79d11aa6f6fb None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:44.953947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b03a-ba03-484b-86a8-79d11aa6f6fb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:44.954439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b03a-ba03-484b-86a8-79d11aa6f6fb None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:44.956971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 258/514] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:44.961951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b03a-ba03-484b-86a8-79d11aa6f6fb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:44.966396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bb07c-1779-4d7a-b693-9e8eb1d95824 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.001234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f426351-4648-4156-a4f3-156942440284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.001869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426351-4648-4156-a4f3-156942440284 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.002153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426351-4648-4156-a4f3-156942440284 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.002427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f426351-4648-4156-a4f3-156942440284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.014129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f426351-4648-4156-a4f3-156942440284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.015614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f426351-4648-4156-a4f3-156942440284 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-5f426351-4648-4156-a4f3-156942440284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:45.031478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:45.032435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:45.033350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:45.033350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:45.033350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:45.038732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 257/515] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:45.040021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.040310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.040585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.058605 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:45.059852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:45.086527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.101764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b03a-ba03-484b-86a8-79d11aa6f6fb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.131778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 258/516] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:45.132624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.151473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.152936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.153215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.153485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.168861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.171790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:45.192960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:45.193937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:45.194934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:45.194934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:45.194934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:45.198859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 259/517] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:45.198859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.217156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.249645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e830ff-4145-4737-aad4-fb257951c65a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.294990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.295754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.296154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.296542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.297619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:45.298030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.300256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.320241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 259/518] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:45.320241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.331984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9665309-99f8-457e-9f52-f05493ab722e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.377209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.377962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.378310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.378656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] PATH_INFO: `/v3/services/network` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.379596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.381756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:45.397960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:45.399197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:45.400072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:45.402976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 260/519] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:45.403822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.416436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.433803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cec26-b3e1-4e68-be79-d98a8117043b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.464155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.470346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.470989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.471272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.471624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.472213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:45.472517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.475950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.485803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 260/520] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:45.486662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.513854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239d26c-48c8-4fef-8e99-2c8f3511b178 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.549826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.550436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.550719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.551021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.551676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.553140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.597320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee78c4-cb19-4b9a-8b64-fbe71f7e7bff None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.631538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.632107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.632388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.632662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.633263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.634664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:45.647253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:45.650964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:45.651884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 261/521] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:45.652703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.658677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 261/522] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:00:45 2022] POST /identity/v3/users => generated 254 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:45.660025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.731906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.740335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.786135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a723ee-e7e7-49e8-b3e5-a372fdadafdb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.787973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e01fc-f1d0-4882-b13f-699acefffd62 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.827070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.828421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.829026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.829454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.830244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] RBAC: Adding query filter params (name=network) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:45.830930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.832080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:45.832683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.832910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:45.833286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:45.833675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:45.838914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 262/523] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:45.840062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.847038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:45.848604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:45.862377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:45.863477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:45.864557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:45.865295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 262/524] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:45.868054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:45.967179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43971e-f73e-42e8-ace7-15cef9e73c5b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:45.989672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:45.997577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ba86-37ff-4978-964c-f5f52117a277 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:46.008341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:46.009270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.009694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.010140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.010988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:46.011523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:46.013232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:46.023012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 263/525] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:46.024364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:46.024808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:46.036178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:46.036931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.037214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.037487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.038185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:46.038492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:46.040013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:46.054979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 263/526] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:46.056300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:46.179168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc09d6-ebf4-4d71-a244-9cf041ec3fd8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:46.221124 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:46.222072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.222550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.223058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.223926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:46.224650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:46.227927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:46.237417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 264/527] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:46.266881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739dc68-cff3-4d7a-bc3f-a5cfefc7c916 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:46.306376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:46.307479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.307976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.308442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.309372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:46.309879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:46.311575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4cd5-d8e5-4658-b507-e4ab17d71de4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:46.327356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 264/528] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:46.333086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:46.394895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:46.449771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69386-e248-4057-a789-a0016d67dac4 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:46.488954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:46.489883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.490355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.490845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.491784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=d256290285d44aafb8062ec2059b16ea) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:46.492315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:46.493912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685a456e-464c-4421-86ce-f11d3fd0ad52 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:46.500591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 265/529] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:46 2022] GET /identity/v3/role_assignments?user.id=d256290285d44aafb8062ec2059b16ea&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:46.988957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93120d3-112d-455f-86eb-8e921e94e144 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.989465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93120d3-112d-455f-86eb-8e921e94e144 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.989542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93120d3-112d-455f-86eb-8e921e94e144 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:46.991138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 265/530] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:46.997389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:46.997811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:46.997889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:47.018723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:47.019949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:47.100256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 266/531] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:47.479535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93120d3-112d-455f-86eb-8e921e94e144 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:47.540853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:47.628802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93120d3-112d-455f-86eb-8e921e94e144 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:47.676842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:47.677473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:47.677753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:47.678026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:47.692108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:47.696909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:47.697996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:47.698935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:47.699623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 266/532] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:47.704243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:47.782562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:47.858656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa800af-a09f-481f-aa8f-a112d310571b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:47.909437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:47.910055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:47.910340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:47.910613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:47.911323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:47.911628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:47.912995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:47.926785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 267/533] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:47.933062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:48.089096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3df699-582f-46b3-a3a1-ed6eb03b9857 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:48.090563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:48.140948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:48.141586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.141869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.142144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.153116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:48.154572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-85c457d0-c4ae-4b6a-a10d-3a7d7f0107ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:48.176025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:48.177011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:48.177901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:48.177901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:48.177901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:48.177901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 267/534] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:48.182968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:48.350051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd0516-78dc-40f3-b060-5da7bc8a4a9e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:48.362640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:48.362807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e789-502a-4850-b2b0-ac823a1a6ed5 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.362807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e789-502a-4850-b2b0-ac823a1a6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.362807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306e789-502a-4850-b2b0-ac823a1a6ed5 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.363414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 268/535] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:48.367773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.368061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.368339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.398630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:48.400006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:48.412391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:48.413303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.413782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.414272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.415172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:48.415670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:48.417797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:48.443100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 268/536] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:48.449567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:48.496996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 269/537] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:48.597882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d13d7-bb94-473c-93f0-3abc7661010f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:48.653535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:48.654172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.654455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.654728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.664519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:48.665955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:48.682151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:48.683196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:48.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 269/538] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:00:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:48.691916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:48.782955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:48.831549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:48.845533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:48.875831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103cfbf-846e-4e00-a1b6-fbb6cb6822d1 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:48.946182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:48.946793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:48.947103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:48.947917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:48.948896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:48.949208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:48.950663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:48.988122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 270/539] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:00:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:48.991358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691d2d2-d256-4ee9-8bcb-4f811262215e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:48.994396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.041779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.042415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.043109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.043385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.054766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7cfa36b5-dfe9-4b27-abbc-0ef7f8192b27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:49.060132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:49.061068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:49.061908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 270/540] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:49.063252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39923e53-b21c-4631-91bb-23b6bb2422d2 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.063252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39923e53-b21c-4631-91bb-23b6bb2422d2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.063252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39923e53-b21c-4631-91bb-23b6bb2422d2 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.064444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 271/541] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:49.065259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ffbc5-8be0-4fb0-aa03-a441d01c619c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.065540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ffbc5-8be0-4fb0-aa03-a441d01c619c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.066017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ffbc5-8be0-4fb0-aa03-a441d01c619c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.073255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 272/542] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:49.074055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ffbc5-8be0-4fb0-aa03-a441d01c619c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.122661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf039db-87dd-4a0e-be60-767dd7af530d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.161615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.162455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.163027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.163308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.169556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.177254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.183454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ffbc5-8be0-4fb0-aa03-a441d01c619c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.222368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.222917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.223204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.223480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.224147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:49.224452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.225880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220a528-3643-4fe9-9de8-e4c864eb26bf None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.238779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 273/543] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:49.239835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7adb4-da8d-4973-bd47-35f0a698e509 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.240124 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7adb4-da8d-4973-bd47-35f0a698e509 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.240403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7adb4-da8d-4973-bd47-35f0a698e509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.241608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.243102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.258095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f2a7adb4-da8d-4973-bd47-35f0a698e509 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:49.259314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f2a7adb4-da8d-4973-bd47-35f0a698e509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:49.329517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 274/544] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:49.330750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f609be-9cb1-4c32-bd5b-1bfa965ade75 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.331066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f609be-9cb1-4c32-bd5b-1bfa965ade75 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.331345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f609be-9cb1-4c32-bd5b-1bfa965ade75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.348106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-88f609be-9cb1-4c32-bd5b-1bfa965ade75 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:49.349291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-88f609be-9cb1-4c32-bd5b-1bfa965ade75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:49.356134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 271/545] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 26 11:00:49.356838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.403838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 275/546] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f394-b7c0-4c9c-8771-52cd1e82ca1b None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f394-b7c0-4c9c-8771-52cd1e82ca1b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821f394-b7c0-4c9c-8771-52cd1e82ca1b None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 276/547] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:49.426842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f394-b7c0-4c9c-8771-52cd1e82ca1b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.439976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.483459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711cee5-e99f-4418-8401-8fa327f8235b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.532777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.533237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821f394-b7c0-4c9c-8771-52cd1e82ca1b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.533670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.534020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.534363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.547980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.549569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-5d61f8e4-6523-4f03-bed0-59d0ee507d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:49.568397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:49.569532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:49.570599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:49.570599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:49.570599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:49.574980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 272/548] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:49.576216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.576501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.576779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.582369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.583413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.583902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.584378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.596205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.597963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:49.613091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:49.614181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:49.615284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:49.615284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:49.615284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:49.615284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 277/549] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:49.615531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.639516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:49.641287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:49.675176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.719023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 273/550] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 26 11:00:49.719248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a4e29-60b0-46c9-b7b1-2c60282e87fe None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.719334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.743134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.757949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.758745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.758884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.759529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.760269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:49.760571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.762004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.779413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 278/551] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:49.779413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.833315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38ad12-4d5e-47c7-b0b6-16f72ed5e532 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.867895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.868499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.868781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.869576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.870222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.871641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.876736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f66104-24d7-4cb2-a29c-0454f92dc01f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:49.899726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:49.910728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:49.911407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:49.911738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:49.912012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:49.912643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:49.912948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:49.914333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:49.925289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 279/552] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:49.926092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:49.972371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 274/553] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:00:49 2022] POST /identity/v3/services => generated 240 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:49.973096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.033511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce1ca-0b5f-4863-9db7-2c42c5e87ce9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.074603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.087534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.088209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.088523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.089313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.089556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.091037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.092751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f67c6-7d4a-45ba-83ac-8c280e786e38 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.129141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.129846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.130057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.130473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.139177 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.140966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:50.152879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:50.153883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 275/554] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:00:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:50.154853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.170894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 280/555] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:00:49 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:50.170894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.227264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.295588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e1ad4-3db5-4dc8-a0f3-02b94e097067 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.324652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790d276-9ca1-4b5f-8cc1-8550a4f0cb65 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.334832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.335557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.335841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.336113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.343482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.344938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:50.358158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:50.359147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 276/556] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:50.360466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.364631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.368209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.368874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.369707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.369985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.385174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:50.393015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:50.425564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 281/557] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:50.426435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.478244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db799a-ff92-42be-9b42-ac205d6b5302 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.517711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.518680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.518680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.518875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.519585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:50.519889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.521316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.526078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.536384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1fa01-6ff0-4775-8991-217be1ed54f7 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.540519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 277/558] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:50.541297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.585097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.585625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.585898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.586166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.586925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:50.591346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.591346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.600563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 282/559] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:50.601371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.657748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.713070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de2a23-eee8-4ded-ba53-7d5d4f413e31 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.720335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fe82-2e8f-47a1-951e-e101082f1959 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.760803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.761028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.761436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.761711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.761980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.762700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.763226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.763719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.764584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:50.765354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.767071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.774994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.776471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.777591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 278/560] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:50.778786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:50.794260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:50.795329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:50.796219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:50.796219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:50.796219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:50.796219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 283/561] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:50.796955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.808171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.898157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69268457-2b8b-474a-b11f-6622e51faa84 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.906387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68502f39-a537-4cbe-bbb7-9f9b9c175dd2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:50.924268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:50.937294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.938209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.938691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.939190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.940124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=03aad2f885bc4dc183e934469676a42f) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:50.940627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.941110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:50.941735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:50.942012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:50.942284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:50.943138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.950082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 279/562] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/role_assignments?user.id=03aad2f885bc4dc183e934469676a42f&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:50.951185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:50.953619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:50.957617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:50.970615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:50.971665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:50.972553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:50.972553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:50.972553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:50.972553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 284/563] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:50.981041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.091225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:51.150513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92765d03-b4f5-4d4c-bc9f-2f655b0f7c04 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.156845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42520eed-96b9-4c8a-88c0-585ccdd77dd1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.193285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.194096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.194380 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.194741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.195523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:51.195918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.197333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.197531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.198161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.198539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.198956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.199719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:51.200126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.201629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.213311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 280/564] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:51.219568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.221611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 285/565] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:51.226198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.280246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:51.336176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47354cf3-813d-4d78-8b08-aa3c6bb911a7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.347551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15e690-0154-4ab8-a166-24797e3b85c0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.370508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:51.389069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.389716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.389999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.390530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.404046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.404671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.404954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.405228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.408674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.410179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.422108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.423608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:51.431190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:51.432114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:51.433016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 281/566] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:51.438763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:51.441159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:51.442087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:51.443063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:51.445713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 286/567] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:51.452418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.635732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c24f7d-b714-46e9-a19a-938bb1d72c6f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.639986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100069e-32ce-4741-8127-ba1b7e7ada79 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.657000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:51.675326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.676257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.686330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.686330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.686330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:51.686330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.686330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.698415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 287/568] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:51.704203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.706202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.710364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.713462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.714040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.714974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:51.715477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.717097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.745621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 282/569] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:51.751198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.779202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:51.881354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6644dc6-3222-429b-ba44-693bd3746c12 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.920755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f7a8a-006a-40c8-9814-b3dfe8527034 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:51.925862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.926738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.927249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:51.927885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:51.948522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:51.951801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:51.978736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:51.980018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:51.980905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:51.980905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:51.980905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:51.985410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 288/570] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:51.992141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:51.996621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:51.997519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:51.999618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.000121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/016e7bb933254672b760d32da32eb6b7/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.027823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=016e7bb933254672b760d32da32eb6b7, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:52.029970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:52.106459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:52.122368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 283/571] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:51 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/016e7bb933254672b760d32da32eb6b7/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:52.126710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa90b6-e27d-4459-b1b8-35dd619036b0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:52.167747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:52.168465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.168755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.169035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.169786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:52.170155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:52.171739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:52.185685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 289/572] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:52.190988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:52.224110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:52.336542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0221fad-01b8-4a56-9fbe-5afaeb9f8a3c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:52.376306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:52.376947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.377223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.377491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.385876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:52.387365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-d98d2bdd-bd10-423e-a85e-5ff6a50ebe1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:52.399791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:52.400833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:52.401727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:52.407164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 284/573] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:52.407956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:52.484738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bcec9-8537-46e3-a7d2-64987dc550e1 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.485043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bcec9-8537-46e3-a7d2-64987dc550e1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.485208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845bcec9-8537-46e3-a7d2-64987dc550e1 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.487679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 285/574] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:52.491853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50781db-082e-4f5d-9024-bf52850e2c59 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.492134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50781db-082e-4f5d-9024-bf52850e2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.492410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50781db-082e-4f5d-9024-bf52850e2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.515901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b50781db-082e-4f5d-9024-bf52850e2c59 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:52.517100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b50781db-082e-4f5d-9024-bf52850e2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:52.564995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa79d9-d220-4f32-922b-f7b6d1e1cd83 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:52.606239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:52.606898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.607187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.607464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.608171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:52.608482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:52.610024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:52.624346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 290/575] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:52.631797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:52.650915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:52.693399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 286/576] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:00:52.751586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9a77-fffb-4166-932f-94c8e017c3d3 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.751871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9a77-fffb-4166-932f-94c8e017c3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.752144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c9a77-fffb-4166-932f-94c8e017c3d3 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.757479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 287/577] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:00:52.758861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.759151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.759724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.774045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d783-0ad7-40b9-a7b8-3e091f5a71cb None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:52.778706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:52.780048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:52.782151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:52.811043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:52.811971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:52.812450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:52.812904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/d256290285d44aafb8062ec2059b16ea/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:52.815090 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=d256290285d44aafb8062ec2059b16ea, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:52.816675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:52.859544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 288/578] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:00:52.923871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 291/579] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:00:52 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/d256290285d44aafb8062ec2059b16ea/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:00:53.022408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.140738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32427da2-d0c0-4fa8-9886-42728b3f4b1a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.177601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.178613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.179139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.179607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] PATH_INFO: `/v3/services/network` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.180396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.182038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:53.191477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:53.192930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:53.193815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.197396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 289/580] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:53.203389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.278861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:53.279158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.282000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:53.334293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6c9ef-47ac-4f42-97b9-0e50c68f06f1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.396835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.397465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.397746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.398020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.398665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] RBAC: Adding query filter params (name=network) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:53.399003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.401005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.406828 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 292/581] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:53.415656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.438872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e76fa-aec1-495b-9f34-d79072b3ae94 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.509755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.510395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.510671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.510971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.511552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.512921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:53.534686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:53.535978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:53.537170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 290/582] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:53.542999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.587181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b9650-ed51-4ecd-b965-6b851feac846 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.617878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:53.623781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.624701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.625150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.625699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.626330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] RBAC: Adding query filter params (type=network) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:53.626661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.628053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7c8a34-8090-474f-b314-2f550bb58ecc None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.633448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 293/583] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:53.635390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835077c0-c5de-48d6-840f-8771ccdbfbc0 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.635676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835077c0-c5de-48d6-840f-8771ccdbfbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.635955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835077c0-c5de-48d6-840f-8771ccdbfbc0 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.637696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 294/584] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:53.642395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835077c0-c5de-48d6-840f-8771ccdbfbc0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.651769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:53.693358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3d20-3382-4824-a2eb-290345a46ff5 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.744484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.744484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.744784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.744863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.745491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:53.745796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.747210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ede6c3-974b-4bc2-ac44-ca7e6ca14f1b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.756674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 291/585] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:53.758030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.758319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.758596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.774683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835077c0-c5de-48d6-840f-8771ccdbfbc0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:53.777983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:53.779090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:53.817809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:53.818748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:53.819288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:53.819762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:53.820661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] RBAC: Adding query filter params (interface=public, service_id=068802b59a344c92a4a83745c4d5e6ae, region_id=RegionOne) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:53.821170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:53.823817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:53.833348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 295/586] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/endpoints?service_id=068802b59a344c92a4a83745c4d5e6ae&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:53.834421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.870843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 292/587] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:00:53.870843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:53.954646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:53.973113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:54.025878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e4dda-098b-4239-9fa4-a45a733d0f5b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.055510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3a59a-0a56-4e8d-8514-5dc0a6301d5e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.098095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.098620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.098988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.099263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.099886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:54.100188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.104128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.114952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 296/588] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:54.122522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:54.124779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.125632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.126105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.126587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.127423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.129171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.136512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 293/589] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:00:53 2022] GET /identity/v3/services => generated 772 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:54.207483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:54.312614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc4da1-7a9e-4eeb-9459-25f4dca5b107 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.370288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:54.381884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.382847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.383318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.383750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.398054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] RBAC: Adding query filter params (interface=public, service_id=dbc4cba6cb264c229c86acacad29736c, region_id=RegionOne) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:54.398054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.398054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.398452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 297/590] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:00:54 2022] GET /identity/v3/endpoints?service_id=dbc4cba6cb264c229c86acacad29736c&interface=public®ion_id=RegionOne => generated 194 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:54.400682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:54.407011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df53455-6a1c-4a24-b09e-462b0470b393 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.440125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:54.456048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.457099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.457471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.457836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.458672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.460447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:54.472920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:54.473909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:54.475255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 294/591] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Oct 26 11:00:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:54.480986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:54.519603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc0a13-3540-4b1a-9cdb-34f8700fafc2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.556055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.559083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.559083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.559083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.559083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.559481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.567657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 298/592] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:00:54 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:54.614114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46146307-2684-41ba-b86f-bfc44505ec4e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.653815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.654780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.655309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.655831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.656724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:54.657252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.659009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb27c-06f0-449a-8660-b6551062f0ed None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.665424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 295/593] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:00:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:54.671848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:54.748900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:54.799116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefd63a-114e-4a1a-86b4-029ffb221662 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:54.803851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb625f-e8d6-4413-abef-8bc5862c8978 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.804145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb625f-e8d6-4413-abef-8bc5862c8978 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.804424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb625f-e8d6-4413-abef-8bc5862c8978 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.809948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 296/594] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:54.811592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.811983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.812249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.833363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:54.834789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:54.836240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:54.836860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:54.837840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:54.838196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:54.839678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:54.839678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:54.840834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:54.845915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 299/595] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:00:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:54.898503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:54.927021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 297/596] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:00:55.202570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:55.308792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142dd46-ac6d-4b31-9c49-074473624296 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:55.325830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:55.344276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:55.344908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:55.345190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:55.353644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:55.356805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-53df5715-0b0f-4303-8343-9508def9bc53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:55.361629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:55.362582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:00:55.363474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:55.366943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 300/597] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:55.369256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:55.537643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe508b-a576-4f63-bd4f-e9bc46142977 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:55.564120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99aa85-ffbe-4286-81c1-20cad798c3de None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:55.565008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99aa85-ffbe-4286-81c1-20cad798c3de None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:55.565467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99aa85-ffbe-4286-81c1-20cad798c3de None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:55.568458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 301/598] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:55.573265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:55.573796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:55.574296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:55.585481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:55.586093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:55.586373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:55.586649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:55.587895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:55.588201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:55.589936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:55.595306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:55.596784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:55.607669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 298/599] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:00:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:55.618625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:55.678925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 302/600] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:00:55.701554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:55.802614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3206558-85f3-4ba1-94ad-8881eab18491 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:55.851522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:55.852152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:55.852429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:55.852723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:55.867948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:55.869936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:55.892226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:55.893264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:55.894610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:55.894610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:55.894610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:55.894610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 299/601] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:55.901075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:55.989569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.019784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86764171-36d6-45c5-90f0-cecc68ed94c1 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.027825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.057109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.057109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.057109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.057109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.058014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:56.058522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.060113 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.076594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 303/602] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:56.078068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.185962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507ed2e-4fdd-458c-a3fa-5597ddfe322f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.218997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.221086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.221958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.222443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.222951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.232492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83767190-2426-4a0d-981e-8a76cb42a81c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.234199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.235889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:56.248257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:56.249266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:00:56.250216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:56.257098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 304/603] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:56.258518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.287189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.307580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.308145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.308468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.308742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.331915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:00:56.337375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:56.338335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:56.339926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 300/604] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:56.344300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.369934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd6e6-4a46-4139-8a0c-07ab64d31abd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.404167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.405107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.405593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.406092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.407059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:56.407574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.409183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.432373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 305/605] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:56.433940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.523210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.539661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18146d-8cd8-4c87-ac3a-df8fc8dfc998 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.557013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3135b-4cb2-469e-a313-fa54af7fbc53 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.582889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.582889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.582889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.582889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.652492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.652707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.652989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.653266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.653926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:56.654226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.655618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.656101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.657853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.674408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 301/606] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:56.683792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.726289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.750363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:56.773706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 306/607] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 26 11:00:56.775919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:56.885354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095448ee-04fb-42c9-af05-8a7653808697 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.905135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936bb18-cdec-4549-9c4b-384788755b0c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:56.931485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.932501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.932984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.933458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.952123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:56.954855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:56.976084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:56.977685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:56.978449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:56.978981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:56.987906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:56.988892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:56.994717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:56.994717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:56.994717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:56.994717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 307/608] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:00:56.998712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.000320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.004260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-d14c194f-8728-441d-8135-8b2d8df4d95a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:57.024353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:57.025928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:57.026858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:57.026858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:57.026858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:57.029349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 302/609] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:57.030887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a70c-d6f9-4f6d-8a0f-7995e31ff8b8 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.031378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a70c-d6f9-4f6d-8a0f-7995e31ff8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.031866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1a70c-d6f9-4f6d-8a0f-7995e31ff8b8 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.033563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 303/610] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:57.034899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1a70c-d6f9-4f6d-8a0f-7995e31ff8b8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.050527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.120685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.123171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97157eba-0cc3-428a-a17f-f6d85e685d72 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.165114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.165643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.165925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.166201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.166873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:57.167187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.168680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c43b6-d256-4f05-854c-1871d1a498ae None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.169003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1a70c-d6f9-4f6d-8a0f-7995e31ff8b8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.189047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 308/611] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:00:57.190347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.190635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.190938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.209812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:57.211165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:57.211656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.213095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.213942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.214420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.216332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:57.216850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.218457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.240674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 304/612] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:57.240674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.300272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 309/613] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:00:57.300272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.351817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.402780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddcba23-419b-412a-8aec-4005c193e675 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.429264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c15c11-a6e8-4b50-82b1-fa9a01948f20 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.440609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.448725 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.449362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.450057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.450349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.459450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.460896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.463096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.463727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.464002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.464268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.474384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.474384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-a417cf59-4060-444d-9083-47023618a6f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:57.480796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:57.481776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:57.482739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:57.484831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 305/614] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:57.486283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eda0e-9111-48f2-8d45-92940a530b22 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.486775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eda0e-9111-48f2-8d45-92940a530b22 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.487308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eda0e-9111-48f2-8d45-92940a530b22 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.488996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 306/615] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-bda8decd-b5b3-42e5-bb2a-9ba5e189dd5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:00:57.493834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:57.494767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 26 11:00:57.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:57.496331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eda0e-9111-48f2-8d45-92940a530b22 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.496415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 310/616] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:57.497148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.497148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.497148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.517944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:57.519150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:57.595465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 311/617] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:00:57.596109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.626079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eda0e-9111-48f2-8d45-92940a530b22 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.642193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.670892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.671508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.671790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.672070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.672779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:57.673090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.674557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.689613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 307/618] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:57.689613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.716756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42888fff-857c-4fc5-b01f-207abfc13a62 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.744882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.758305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.759076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.759353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.759619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.760340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:57.760642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.762055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a037bc-ecd7-4c0e-93d9-04771b93ad14 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 312/619] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767118e-df73-47e8-8de8-fca9bb1ac1f2 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767118e-df73-47e8-8de8-fca9bb1ac1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767118e-df73-47e8-8de8-fca9bb1ac1f2 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 313/620] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:00:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:00:57.785541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767118e-df73-47e8-8de8-fca9bb1ac1f2 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.802931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a1496-bdf3-4959-a8d9-a9451cf492a9 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.841146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.841797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.842101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.842347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] PATH_INFO: `/v3/services/network` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.842916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.844287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:57.853497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:57.854558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:57.855451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 308/621] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:57.856270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:57.869961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.894922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767118e-df73-47e8-8de8-fca9bb1ac1f2 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.928586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.929174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.929447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.929714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.930421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=016e7bb933254672b760d32da32eb6b7) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:57.931285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:57.932820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611b430-9e6d-49c1-836a-268702074f26 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:57.938631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 314/622] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/role_assignments?user.id=016e7bb933254672b760d32da32eb6b7&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:57.939818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:57.940042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:57.940310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:57.958846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:00:57.959052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e30d74-009d-4d83-9cfc-749f93f824bd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:57.960076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:00:57.975689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:57.998760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:57.999729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.000007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.000278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.038909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 315/623] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:00:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:00:58.047830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.093293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.094764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.139381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.180267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49783b7-168c-4df2-b3be-e9a2ab863f25 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.216553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 309/624] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:00:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:00:58.217445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.220896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.221756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.222247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.222738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.223598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:58.224104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.225672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.231767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 316/625] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:58.232791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.233912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.337808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e30f8-198f-4f6e-b07d-03243aaf3dfa None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.343511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992c3cd-5730-4b2c-b278-65aada93cfa1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.373332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.373959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.374234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.374504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.375104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.376227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.376454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.381094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.381722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.382002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.382273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:58.385467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:58.392733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:00:58.393579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 310/626] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:58.394387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.395934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.397382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:00:58.412985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:58.413975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:00:58.414957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:00:58.414957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:00:58.414957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:58.419545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 317/627] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:58.420233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.477741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.499796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f41976-470e-4048-a73e-d974aaab1d19 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.528658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532e2a0-fc9e-42ac-8b6f-044bacc1c89d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.539348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.540151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.540437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.540713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.541340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:58.541645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.543035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.549088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 311/628] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:58.549875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.568177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.568788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.569055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.569324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.569998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.571877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.643593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 318/629] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:00:58 2022] POST /identity/v3/services => generated 269 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:58.644373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.659553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a78b76-53bc-410c-8840-c48f55e1bee7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.705361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.719892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.724458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.724959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.727651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.727651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:58.727651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.728307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.738981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 312/630] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:58.743365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.746962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.784696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d2345-2bc2-467b-87a1-4a408cb2d7ec None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.826520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.827263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.827611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.827950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.828665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:58.829032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.830493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.845271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 319/631] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:58.846285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:58.888627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7623-1ca6-45c7-bdca-91599cf6fe3b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.933734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.934348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.934626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.934918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:58.935576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:58.937485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:58.964417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac56b-b6d0-4065-95cb-3f465daf9acb None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:58.967710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:58.998390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:58.998951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:58.999233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:58.999523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:59.000147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:59.000450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:59.001808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:59.007319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 320/632] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Oct 26 11:00:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:59.012768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:59.042278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 313/633] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:00:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:59.043495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:59.124544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:59.188609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949574d6-de92-4f4b-aefe-1dbc1bdf75db None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:59.201091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef388f-d5c3-4126-ad87-12f98861a181 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:59.232007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:59.233165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:59.233658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:59.234128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:59.235217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:59.236840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:59.274207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:59.275874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:59.276378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:59.276896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:59.285258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:59.287145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:00:59.300508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:00:59.303614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:00:59.304513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 321/634] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:00:59.310898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:59.342306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 314/635] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:00:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:00:59.343877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:59.431746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc182c2-9b38-4056-b1ea-6ec0ba45d308 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:59.482765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:59.483839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:59.484318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:59.484805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:59.485720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:59.486221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:59.487899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40321ff-d085-404a-9cab-eba9dff38eb7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:59.502702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 322/636] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:00:59.509705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:00:59.642009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b525e0-7697-4eeb-835c-ef60da9a7460 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:00:59.653252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:00:59.676984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:00:59.678233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:00:59.678660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:00:59.679128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:00:59.679983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=d256290285d44aafb8062ec2059b16ea) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:00:59.680446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:00:59.682511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0063d-3f76-4614-b7eb-2d06c2b0e48e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:00:59.689384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 315/637] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:00:59 2022] GET /identity/v3/role_assignments?user.id=d256290285d44aafb8062ec2059b16ea&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:00.272362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c463e-4d74-4d0d-a30b-f26b3ec2f2bb None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.272904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c463e-4d74-4d0d-a30b-f26b3ec2f2bb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.272983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c463e-4d74-4d0d-a30b-f26b3ec2f2bb None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.274568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 323/638] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:00.279651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.279936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.280210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.298986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:00.300162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:00.370927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 316/639] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:00.645590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c463e-4d74-4d0d-a30b-f26b3ec2f2bb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:00.735195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:00.751809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c463e-4d74-4d0d-a30b-f26b3ec2f2bb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:00.787811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:00.788472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.788748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.789015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.801493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-36039d76-f60b-4940-80c7-e34294a61f78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:00.806499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:00.807544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:00.808396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 324/640] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:01:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:00.814096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:00.908397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c346-d795-4abf-b4fc-21746f610721 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.908584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c346-d795-4abf-b4fc-21746f610721 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.908861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9c346-d795-4abf-b4fc-21746f610721 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.910431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 325/641] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:00.915395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.915683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.915962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.917755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd583a-eff3-4b12-8e07-f7ee4a2d5d29 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:00.921163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:00.934214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:00.935449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:00.956322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:00.957168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:00.957626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:00.958206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:00.959381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:00.959696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:00.961543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:00.972694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 317/642] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Oct 26 11:01:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:00.979233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.018843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 326/643] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:01.097313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874b584-82e5-491d-94b8-ac52689e284c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.132416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.133038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.133314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.133581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.144803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.146300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:01.160200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:01.161134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:01.162315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:01.162315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:01.162315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:01.162315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 318/644] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:01.167254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.249331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.275733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce402c-d5da-44b7-afae-3b6f601ee848 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.301468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.314016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.314636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.315059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.315387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.316217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:01.316569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.318632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.319706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.338989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 327/645] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:01.340602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.415844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662be47-dd6d-430e-9355-ed6161a1ad29 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.451309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.452057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.452338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.452608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.453471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.454622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.462686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e27b-d3cd-4442-9291-b8f7cd1f8005 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:01.463704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:01.464615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:01.465515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 319/646] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:01.472553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.513907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.515657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.515938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.516207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.518205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.524865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.526285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:01:01.538136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:01.539171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:01:01.540705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:01.544559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 328/647] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:01.545475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.572595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.601628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34951845-7cf6-4b81-aa2e-82713c426cbd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.655024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.655763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.656041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.656567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.657634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:01.658082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.659869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.666739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 320/648] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:01.666913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3a409-118d-432b-b457-c3f911e71f39 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.672435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.708253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.709225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.709561 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.709903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.710630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:01.710966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.712967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dcbd2-7509-495f-a805-74f2b0e91905 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.737075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 329/649] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:01.738306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0ed6-4371-4bd2-bf74-45e0babd559c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.738584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0ed6-4371-4bd2-bf74-45e0babd559c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.738932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a0ed6-4371-4bd2-bf74-45e0babd559c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.744709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 330/650] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:01.744709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a0ed6-4371-4bd2-bf74-45e0babd559c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.789062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697b5eb-6654-458e-b9e9-3c8a5f12e796 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.800900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.825378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.826025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.826458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.826734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.827409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:01.827712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.829118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222afa7-0b6f-45e1-8312-903fde04db11 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.835420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 321/651] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:01.836493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.836780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.837053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.856784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:01.858001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:01.861089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:01.861385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a0ed6-4371-4bd2-bf74-45e0babd559c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:01.897804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:01.898428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:01.898703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:01.899003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:01.909080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:01.910523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:01.926884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:01.933276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:01.934470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:01.934470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:01.934470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:01.934470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 331/652] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:01.934470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:01.945334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 322/653] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:01.945992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.031900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344290b-bd8f-40ee-b526-39cbc4aad3ee None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.035530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.059138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509e30-bbea-4da7-a70c-c49fdd604176 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.067819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.068816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.069388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.069717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.070409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] RBAC: Adding query filter params (interface=public, service_id=a90783e55c2d47699d5200a9fd261f51, region_id=RegionOne) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:02.070718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.072630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b56752-42ad-472b-a5c3-a9abd8ae3532 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.077614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 332/654] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/endpoints?service_id=a90783e55c2d47699d5200a9fd261f51&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:02.078747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c8c5-7b67-418a-af95-5b69e89308b7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.079055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c8c5-7b67-418a-af95-5b69e89308b7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.079321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63c8c5-7b67-418a-af95-5b69e89308b7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.080657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 333/655] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:02.085277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63c8c5-7b67-418a-af95-5b69e89308b7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.100162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.101044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.101492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.101936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.102971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:02.103581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.105107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2901cb-12c2-41d2-aca8-c045bf07b38b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.115049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.123118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 323/656] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:02.128243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.128243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.128243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.143621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:02.144794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:02.187552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63c8c5-7b67-418a-af95-5b69e89308b7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.219331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 324/657] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:02.220172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.220276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.220887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.221170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.221444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.222038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.223429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.238862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 334/658] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services => generated 1029 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:01:02.238862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.281468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.354063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec44e40-9c2b-4b42-a857-f1b003b42054 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.375434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e097a-bd76-4d00-8d98-7ed9ba438bfe None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.386881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.394084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.394797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.395109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.395381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.402777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.404937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:01:02.418064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:02.419475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:02.421684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:02.422981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4160-2021-4901-ae48-88c04163772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.422981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4160-2021-4901-ae48-88c04163772d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.422981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4160-2021-4901-ae48-88c04163772d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.422981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e4160-2021-4901-ae48-88c04163772d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.423668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4160-2021-4901-ae48-88c04163772d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.423668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e4160-2021-4901-ae48-88c04163772d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.424176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 325/659] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:02.425018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-698e4160-2021-4901-ae48-88c04163772d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:02.435959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:02.437027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:02.438992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 335/660] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:02.442750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4160-2021-4901-ae48-88c04163772d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.543715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.544058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90146425-2aff-4970-b6de-7533e3719881 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.550595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e4160-2021-4901-ae48-88c04163772d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.578269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.579237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.579727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.580200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.581074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:02.581567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.583268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.588021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.588631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.588918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.589193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.589818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:02.590123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.591566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.600817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 326/661] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:02.600817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.601081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 336/662] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:02.602755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.651532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.703964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feae26-7585-4934-b1d4-4a406bf2e7e7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.706329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c13e6e-49a4-4247-be53-1227c5d8f6bf None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.748913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.749585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.749873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.750148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/016e7bb933254672b760d32da32eb6b7/roles/376b0ac2bb5046a48b58293c4a41f632` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.750738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.751407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.751730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.752021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.752379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=016e7bb933254672b760d32da32eb6b7, role_id=376b0ac2bb5046a48b58293c4a41f632)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.752609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.753803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.753997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:02.763564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:02.768097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:02.769127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 327/663] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:02.769936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.850100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 337/664] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:01:02 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/016e7bb933254672b760d32da32eb6b7/roles/376b0ac2bb5046a48b58293c4a41f632 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:01:02.851304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:02.878140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.886444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cef71c-76fa-431f-abfb-19387b01cb06 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:02.907572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:02.944842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:02.945897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:02.947375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:02.955175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:02.955175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:02.955175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:02.955175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:02.955554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 328/665] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:02.981985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18745c0-0981-4601-8771-d960c2df2749 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:03.027301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:03.028339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:03.028829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:03.029452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:03.030440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:03.030988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:03.032651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25805720-0057-4a0d-a536-cdac69d6ca73 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:03.039104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 338/666] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:03.045362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:03.166482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e9382-86e3-49a9-9c0d-217efe141f19 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:03.206781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:03.207820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:03.208309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:03.208789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:03.209618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:03.210128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:03.211733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78278a72-a15a-485a-b0a7-db5cd8769917 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:03.217503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 329/667] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:04.429273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607fb6-cbeb-40d9-916c-6e49a904a29d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:04.429786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607fb6-cbeb-40d9-916c-6e49a904a29d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:04.429786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6607fb6-cbeb-40d9-916c-6e49a904a29d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:04.431471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 339/668] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:04.436680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:04.437079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:04.437248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:04.456576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:04.457758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:04.495158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:04.531088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 330/669] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:04.797771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607fb6-cbeb-40d9-916c-6e49a904a29d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:04.886481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:04.904353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6607fb6-cbeb-40d9-916c-6e49a904a29d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:04.941496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:04.942160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:04.942442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:04.942722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:04.943378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:04.945054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-99166ed5-30b7-43cf-a359-6c0b0792cf38 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:04.955640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:04.956661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:04.957556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 340/670] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Oct 26 11:01:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:04.963010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.051621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c337c-29c2-4c4d-bebb-848ac8717d19 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.051867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c337c-29c2-4c4d-bebb-848ac8717d19 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.052259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c337c-29c2-4c4d-bebb-848ac8717d19 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.058554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 341/671] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:05.060503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bbd93-b0e1-46c2-b612-579ac0b9ce57 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.060883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bbd93-b0e1-46c2-b612-579ac0b9ce57 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.061304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bbd93-b0e1-46c2-b612-579ac0b9ce57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.077001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab970a13-bc42-4a24-b9ee-7ff74dad4785 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.082354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a22bbd93-b0e1-46c2-b612-579ac0b9ce57 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:05.083692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a22bbd93-b0e1-46c2-b612-579ac0b9ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:05.114808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.115478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.115756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.116029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.116671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:05.116976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.118671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.124589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 331/672] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:05.130382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.172512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 342/673] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:05.175263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.242896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913e251-9f82-4c79-88c5-bc0a2d18460c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.243532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913e251-9f82-4c79-88c5-bc0a2d18460c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.244085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913e251-9f82-4c79-88c5-bc0a2d18460c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.244842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5080f6-c9ed-4960-8b10-c57dce72b2ba None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.246100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 343/674] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:05.251544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.252069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.252605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.272912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:05.272912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:05.287712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.288476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.288762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.289032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.290082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:05.290682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.292265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.298709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 332/675] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:05.304928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.358997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 344/676] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:05.426085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.429765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8418715-d333-4bd3-a7d7-05103a6a3816 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.444523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.468144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.468747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.469025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.469134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.469300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.469963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.471361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e753a1c-8a89-4be7-a01d-063ff0036e70 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.545767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6e9ac-83e6-4b46-8f26-fe3db347bff8 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.569572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 333/677] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:01:05.580594 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9719a4-19c4-4b5f-ac39-f74915906419 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.580908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9719a4-19c4-4b5f-ac39-f74915906419 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.581148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9719a4-19c4-4b5f-ac39-f74915906419 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.582487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 334/678] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:05.588354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.588649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.588897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.591411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.592055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.592334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.592606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.604616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:05.610847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:05.611910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:05.612894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:05.617312 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 345/679] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:01:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:05.618359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.627409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:05.629116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:05.691697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.719471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 335/680] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:05.720815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.739760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796e982-dc41-4775-b035-5355fa8691c5 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.789502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.789999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.790383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.790702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.791416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:05.791728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.793119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.803709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 346/681] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Oct 26 11:01:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:05.810252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:05.834203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.855655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60ee20f-6613-4d42-ad20-0c34aac01284 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.903715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.904334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.904619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.904893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.906088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.907994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.932322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c97fd8f-c629-43d8-a5fd-831c67df49ca None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:05.957885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:05.966480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:05.967144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:05.967421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:05.967690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:05.974912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 336/682] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:05.983903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:05.985449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:05.999766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:06.004075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:06.007209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:06.008107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:06.008107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:06.008107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:06.008107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 347/683] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:06.015365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:06.139156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09447171-83ee-4e47-bde8-5a208ab0f7a1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:06.155125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccc9e1-0660-4bb9-95cf-96b285d74714 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:06.186764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:06.187412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:06.187860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:06.188143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:06.189130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:06.189586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:06.191361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:06.197819 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:06.198948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:06.199235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:06.199513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:06.200624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:06.202278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:06.203509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 348/684] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:06.208582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:06.256361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:06.276685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 337/685] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:05 2022] POST /identity/v3/services => generated 241 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:01:06.345382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f733b-2b87-4def-b28f-404701034afd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:06.392290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:06.392856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:06.393131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:06.393402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:06.402229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:06.403741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:01:06.417583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:06.418645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:06.419683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 349/686] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:01:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:06.426670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:06.429928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:06.540328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9b6de-2b20-4288-ac72-344747d7c220 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:06.575550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:06.576403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:06.576403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:06.576691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:06.577423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:06.577728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:06.579223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:06.601719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 338/687] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Oct 26 11:01:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:06.607861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:06.711187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:06.718253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93212342-a729-4fea-a962-5d5e64716f5e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:06.752710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:06.753769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:06.754298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:06.755088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:06.825231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:06.826685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:06.926485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:06.935382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 350/688] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:01:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:01:06.941819 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:07.004955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:07.049137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102fa29-ee7a-4a09-9367-f55aef029f68 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:07.085262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:07.086231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.086727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.087243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.097787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:07.099525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:07.114508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:07.115499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:07.116465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:07.116465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:07.116465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:07.118451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 339/689] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:07.124753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:07.226141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0060f051-2247-4124-a3dc-cabee11081c8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:07.262238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:07.263248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.263730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.264260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.265135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:07.265914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:07.267546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:07.281240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 351/690] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:07.287033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:07.390984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda3779-68f9-4fda-90e7-9c9b324935bb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:07.404501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:07.426432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:07.427370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.427874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.428358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.435865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:07.437502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:01:07.450062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:07.450993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:07.451887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:07.454328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 340/691] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:07.460270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:07.519334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:07.566483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd014720-3849-4b03-96f8-40345bc3bd73 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:07.602114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:07.603740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.604185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.604640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.605539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:07.606003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:07.607849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd6853-a2b6-48d0-a13f-5055229fc68b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:07.622249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 352/692] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:07.628074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:07.735825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab12b4-763d-4ca5-a615-c722af866a71 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:07.772277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:07.772795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.773069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.773336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.774047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] RBAC: Adding query filter params (role.id=376b0ac2bb5046a48b58293c4a41f632, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=016e7bb933254672b760d32da32eb6b7) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:07.774353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:07.775894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae44a6-52ea-4c18-89c7-453acff3c275 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:07.782183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 341/693] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:01:07 2022] GET /identity/v3/role_assignments?user.id=016e7bb933254672b760d32da32eb6b7&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=376b0ac2bb5046a48b58293c4a41f632 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:07.952506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80ca2-c022-4dc7-8928-3e2e95bb1ebf None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.952833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80ca2-c022-4dc7-8928-3e2e95bb1ebf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.953054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc80ca2-c022-4dc7-8928-3e2e95bb1ebf None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.954595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 353/694] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:07.960276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c114af-7421-48c6-b595-33940b48908e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:07.960813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c114af-7421-48c6-b595-33940b48908e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:07.961296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c114af-7421-48c6-b595-33940b48908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:07.981237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-17c114af-7421-48c6-b595-33940b48908e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:07.982659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-17c114af-7421-48c6-b595-33940b48908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:08.013651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:08.054473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 342/695] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:08.322987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954027f-ecba-41a6-abd3-ffc734a3670d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.323200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954027f-ecba-41a6-abd3-ffc734a3670d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.323482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954027f-ecba-41a6-abd3-ffc734a3670d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.325114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 354/696] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:08.330384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.330679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.330997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.333257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954027f-ecba-41a6-abd3-ffc734a3670d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.352364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:08.354042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:08.440810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 343/697] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:08.458570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954027f-ecba-41a6-abd3-ffc734a3670d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:08.478285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:08.495815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:08.496453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.496727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.496997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.497574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:08.498955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-cdb47af2-d0a6-40b2-94db-9185f36532a1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:08.508942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:08.510587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:08.511483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:08.512227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 355/698] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:08.519504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.543063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f2373-fa31-40fd-ad2c-6a1acc08f400 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.543685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f2373-fa31-40fd-ad2c-6a1acc08f400 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.544165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f2373-fa31-40fd-ad2c-6a1acc08f400 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.545980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 356/699] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:08.551029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.551560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.552019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.581622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:08.583139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:08.637105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:08.646207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691ced9-d5ab-4520-88be-44fa23a6bacd None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:08.659202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 357/700] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:08.685019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:08.685883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.686354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.686868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.687677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:08.688163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:08.689706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:08.695137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 344/701] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:08.701202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.743283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.829111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601dbe61-1941-4256-a6b8-2fdd4c5cb88f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:08.840824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:08.871215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:08.872265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.872817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.873357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.874156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022846c8-0353-449e-9cd8-4829e1992472 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:08.874388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:08.874933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:08.877673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:08.883471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 358/702] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:08.919684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:08.920707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:08.921332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:08.921830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:08.922724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:08.924475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:08.934282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:08.935971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:08.936973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:08.937838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:08.940396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 345/703] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:08.946611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:08.975070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.054927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72eec3c-f7d7-4e10-b820-34cd90bf1b14 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.070801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de154c0-c3bf-4d8e-b32b-06a7146baaeb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.102371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.102989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.103303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.103574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.105746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.107154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:09.117599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:09.118653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:09.119605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 359/704] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:09.126188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.128444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.129058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.129333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.129601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.130276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.130581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.132049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.142568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 346/705] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:09.143645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.175134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.253173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc49bc6-7637-444b-a974-3bc55be6d9de None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.253898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.269396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c8005-ce56-441e-832b-64ada2711b1b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.293647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.294605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.295109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.295582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.296420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.297108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.299361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.310077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 360/706] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:09.311079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.315855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.317115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.317799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.318501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.319554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.320059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.321738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.333017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 347/707] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:09.334224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.420793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.424386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88d3a0-3b4f-4edd-920b-ccc1502b8db1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.451691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff39b8-8c2b-4360-9c4f-06157f029168 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.465938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.466730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.467101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.467437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.468224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.468600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.470019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.476558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 361/708] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:09.482314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.495403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.496016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.496320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.496666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.497368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] RBAC: Adding query filter params (interface=public, service_id=61acf35cf6d94bb6b1982b793a68c570, region_id=RegionOne) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.497679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.499173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.512425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 348/709] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/endpoints?service_id=61acf35cf6d94bb6b1982b793a68c570&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:09.513293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.532780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.604890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4139c-c05d-4eaf-8050-4833718f9f21 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.622924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705c371-1b0a-4b77-aee3-df4f40704136 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.652091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.653145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.653427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.653705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.654729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] RBAC: Adding query filter params (interface=public, service_id=3abf1bd590ce4a0899f35bed4910ce79, region_id=RegionOne) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:09.655208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.656683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.661167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.661859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.662206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.662416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 362/710] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/endpoints?service_id=3abf1bd590ce4a0899f35bed4910ce79&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:09.667516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.668168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.668507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:09.669762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4ea0f-15ae-42f5-9444-2161e7ec834f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.682487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 349/711] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:01:09.701386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:09.792938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a8156-9e77-4b79-b5c4-a2d61cf3eea3 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:09.833486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:09.834638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:09.835134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:09.835594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:09.836501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:09.838249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b006336-7c1a-4f5c-82bf-61a50d97c5b1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:09.845942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 363/712] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:09 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:01:10.052360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd01653-999c-4e93-9599-1490dc673492 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:10.052562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd01653-999c-4e93-9599-1490dc673492 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:10.052834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd01653-999c-4e93-9599-1490dc673492 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:10.054465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 350/713] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:10.060341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:10.060632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:10.060913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:10.079549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:10.080742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:10.153119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 364/714] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:10.424060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd01653-999c-4e93-9599-1490dc673492 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:10.509189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:10.531680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd01653-999c-4e93-9599-1490dc673492 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:10.567829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:10.568463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:10.568749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:10.569024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] PATH_INFO: `/v3/services/image` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:10.569605 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:10.571157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:10.581381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:10.582418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:10.583315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 351/715] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Oct 26 11:01:10 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:10.589483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:10.616320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:10.697586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5efe82-4301-4c8c-882d-9841bc1c83ff None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:10.737423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:10.738027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:10.738301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:10.738569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:10.739217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:10.739516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:10.740864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868f9f-2b50-4ffd-8587-65348e75a8b1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:10.746729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 365/716] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:10.758870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:10.862572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7edd68-78bd-4982-941a-5975a3bc2410 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:10.896233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:10.896946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:10.897300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:10.897651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:10.898350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] RBAC: Adding query filter params (type=image) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:10.898731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:10.900775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c61b2-06de-452b-97db-2b407c8f5a4c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:10.905595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 352/717] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:11.131401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb946-4035-4fef-88ad-4d022fb58e5a None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:11.131786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb946-4035-4fef-88ad-4d022fb58e5a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:11.131977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb946-4035-4fef-88ad-4d022fb58e5a None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:11.133564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 366/718] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:11.139128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494959d1-7019-434a-b96f-ecef00b29534 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:11.139419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494959d1-7019-434a-b96f-ecef00b29534 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:11.139698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494959d1-7019-434a-b96f-ecef00b29534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:11.158597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-494959d1-7019-434a-b96f-ecef00b29534 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:11.159835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-494959d1-7019-434a-b96f-ecef00b29534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:11.161848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:11.234956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 353/719] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:11.495191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfb946-4035-4fef-88ad-4d022fb58e5a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:11.589063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:11.605597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfb946-4035-4fef-88ad-4d022fb58e5a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:11.644942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:11.645561 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:11.645843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:11.646125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:11.646795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:11.648191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66eb367-cb20-4600-a452-f7139b96895c None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:11.720144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 367/720] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:11 2022] POST /identity/v3/services => generated 242 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:11.944401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b358af6-01f1-44f8-9acd-cf57c570413f None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:11.944903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b358af6-01f1-44f8-9acd-cf57c570413f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:11.944980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b358af6-01f1-44f8-9acd-cf57c570413f None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:11.946550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 354/721] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:11.951835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538f38-e3ed-4220-86e9-3f95e23c90b0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:11.952126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538f38-e3ed-4220-86e9-3f95e23c90b0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:11.952405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20538f38-e3ed-4220-86e9-3f95e23c90b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:11.971768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-20538f38-e3ed-4220-86e9-3f95e23c90b0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:11.972961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-20538f38-e3ed-4220-86e9-3f95e23c90b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:12.052201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 368/722] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:12.186615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe289-a9bc-43af-a81b-53ecb02b358a None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.186999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe289-a9bc-43af-a81b-53ecb02b358a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.187201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfe289-a9bc-43af-a81b-53ecb02b358a None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.188781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 355/723] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:12.194060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.194350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.194629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.213362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:12.214559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:12.286436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 369/724] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:12.327067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe289-a9bc-43af-a81b-53ecb02b358a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.432411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfe289-a9bc-43af-a81b-53ecb02b358a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:12.470851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:12.470851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.470851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.470851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.471266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:12.471266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:12.480480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:12.481478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.482344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 356/725] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:12.489542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.494296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:12.581418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.584299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:12.595974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18511534-5388-4d6d-954e-5a9af9c44f7a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:12.638507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:12.639160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.639445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.639982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.640442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:12.641014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:12.642426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:12.648084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 370/726] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:12.654036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.701183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5df10a-d16a-4436-8301-2b989fd983e3 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:12.742769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:12.745012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:12.746075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.746075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.746250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.746901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:12.752699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:12.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:12.758752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:12.760149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 357/727] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:12.765374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.792842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f4f18-7b67-4e0e-9eee-f5b1a6cacf16 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:12.837343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:12.838492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.838693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.839016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.839636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:12.839936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:12.841349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:12.844735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:12.851210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 371/728] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:12.852205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.895042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59300fd-ba5b-4995-9361-62d3a82a78dd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:12.932679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:12.933335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:12.933614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:12.933884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:12.935062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:12.935062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:12.936344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:12.946126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 358/729] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:12.947887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:12.971022 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40291bef-8243-4641-9d7e-7c335cdc6b03 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:13.007335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:13.016044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:13.017031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.022089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.022089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.022089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:13.022089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:13.062704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0262ba-d379-4642-90d1-afe54d02713b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:13.094714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:13.100882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:13.101536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.101852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.102128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.102758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:13.103114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:13.105078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3a39c-4562-4e7c-85ca-235eb4273d23 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:13.113764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 372/730] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:13.116813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 359/731] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:13.123204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:13.147937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd3fc-1df8-4fed-a0b2-3653680aecc0 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.148269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd3fc-1df8-4fed-a0b2-3653680aecc0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.148500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cd3fc-1df8-4fed-a0b2-3653680aecc0 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.149895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 360/732] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:13.156254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.156573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.156821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.178238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:13.179486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:13.260977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea10d2-9ccc-497e-92e7-51b8f5d074d9 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:13.262649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 361/733] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:13.300511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:13.301136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.301412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.301681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.302370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:13.303783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fcc7d-17c6-42f2-8346-a628426f4dec None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:13.382651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 373/734] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:13.538430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:13.647675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42428039-091c-4c2c-8acc-46f20738c82e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:13.672949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:13.689305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:13.689938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:13.690202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:13.690476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:13.691199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:13.692569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144440dd-e41d-4868-9d44-a787b33c06d2 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:13.769805 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 362/735] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:13 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:14.016213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3477-d66c-45b0-b0f0-9c13ef38edb6 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:14.016725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3477-d66c-45b0-b0f0-9c13ef38edb6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:14.016725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d3477-d66c-45b0-b0f0-9c13ef38edb6 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:14.018293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 374/736] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:14.023870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:14.024287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:14.024514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:14.044382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:14.045571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:14.129591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 363/737] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:14 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:14.393838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d3477-d66c-45b0-b0f0-9c13ef38edb6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:14.438421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:14.500225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d3477-d66c-45b0-b0f0-9c13ef38edb6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:14.535798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:14.536793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:14.537268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:14.537753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:14.538536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:14.540169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:14.549386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:14.550398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:14.551277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:14.554011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 375/738] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:14.560281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:14.664365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f0027-9302-4ab2-b592-b051c3640e3e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:14.672715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:14.703418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:14.704027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:14.704305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:14.704578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:14.705203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:14.705509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:14.706902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:14.716577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 364/739] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:14.718041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:14.835729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0baf1-1b61-4d91-8c12-2f8ece4f1df6 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:14.837503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:14.871566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:14.872436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:14.872894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:14.873345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:14.874221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:14.874701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:14.876238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:14.881584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 376/740] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:14.887834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:14.989643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95b31-6cd6-4921-bec2-6c3f0a1bb6f7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:15.024943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:15.025600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.025857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.026131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.026851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] RBAC: Adding query filter params (interface=public, service_id=ad952234b5d449b3a99c23d1960eb7bd, region_id=RegionOne) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:15.027166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:15.028597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a8f09-0628-4156-9a75-4e6e80e99c77 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:15.038035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 365/741] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:01:14 2022] GET /identity/v3/endpoints?service_id=ad952234b5d449b3a99c23d1960eb7bd&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:15.040080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:15.144504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1d1a-f5ed-49ea-b9c2-59144923b05b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:15.178604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:15.179234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.179509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.179776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.180374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:15.182335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cab713-a399-4699-bba6-bc26f23347bf None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:15.189333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 377/742] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:15 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:01:15.460712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079aee-56af-4fab-ba6f-37ae4e1f04ec None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.461201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079aee-56af-4fab-ba6f-37ae4e1f04ec None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.461576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f079aee-56af-4fab-ba6f-37ae4e1f04ec None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.463852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 366/743] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:15.469233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c584b9-af4a-42a0-b2b6-0a87bf8e79da None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.469631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c584b9-af4a-42a0-b2b6-0a87bf8e79da None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.470021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c584b9-af4a-42a0-b2b6-0a87bf8e79da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.489120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a6c584b9-af4a-42a0-b2b6-0a87bf8e79da None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:15.490319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a6c584b9-af4a-42a0-b2b6-0a87bf8e79da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:15.515691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:15.559539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 378/744] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:15.684002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7c92c-b862-4f5e-bc0e-b3b0d20afda0 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.684640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7c92c-b862-4f5e-bc0e-b3b0d20afda0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.685148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7c92c-b862-4f5e-bc0e-b3b0d20afda0 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.686979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 367/745] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:15.692546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e66cf-ee46-4515-b66e-04c842b63e22 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.693064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e66cf-ee46-4515-b66e-04c842b63e22 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.693556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e66cf-ee46-4515-b66e-04c842b63e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.712073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-be9e66cf-ee46-4515-b66e-04c842b63e22 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:15.713736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-be9e66cf-ee46-4515-b66e-04c842b63e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:15.794563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 379/746] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:15.844921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7c92c-b862-4f5e-bc0e-b3b0d20afda0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:15.884248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:15.954564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7c92c-b862-4f5e-bc0e-b3b0d20afda0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:15.957162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9b18-d610-43be-8a65-5b5b271d7570 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.957449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9b18-d610-43be-8a65-5b5b271d7570 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.957724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc9b18-d610-43be-8a65-5b5b271d7570 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.959331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 380/747] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:15.964289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.964508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.964778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.984393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:15.985692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:15.991015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:15.991630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:15.991908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:15.992181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:15.992820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:15.994306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:15.997415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING keystone.server.flask.common [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 26 11:01:16.052926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.sql.core [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:16.052926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 11:01:16.052926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [parameters: {'id': 'bbe98c8a8254414b8bbdcc4d358281bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 26 11:01:16.052926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81744) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbe98c8a8254414b8bbdcc4d358281bc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbe98c8a8254414b8bbdcc4d358281bc. Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:16.062191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 11:01:16.063039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application yield resource Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise self.value Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application session.commit() Oct 26 11:01:16.064006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.session.flush() Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application compat.raise_( Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 11:01:16.064928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 11:01:16.065854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:16.066692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [parameters: {'id': 'bbe98c8a8254414b8bbdcc4d358281bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.069617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:16.070785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbe98c8a8254414b8bbdcc4d358281bc. Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.079914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 368/748] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Oct 26 11:01:15 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:16.080479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 381/749] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:15 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:16.080883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.087478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.132741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.191293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.204242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75755c9d-aceb-478f-ad4a-7926650e39c1 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.232495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abcf29-e751-44cd-ba83-707291c36155 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.252138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.252911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.253253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.253589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.263707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.266304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.271658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.272492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.272961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.273470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.274240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.276520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:01:16.281235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:16.282409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:16.283652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:16.288391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 369/750] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:16.289260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:16.289550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:16.290770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:16.292397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.295283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 382/751] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:16.302849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.395682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.403102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8b18-f938-42c7-af21-802796d57078 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.413611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb0cff-a7a9-40a3-9cb5-e456db3f311e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.436577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.437182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.437458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.437730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.438391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:16.438690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.440132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.449150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.449922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.450184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.450586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.451613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:16.451929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.453265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.462783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 383/752] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:16.462783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.463650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 370/753] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:16.464977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.499292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.586357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e76c3f-793c-4f70-a249-afae09b18003 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.601177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ca392-916e-4950-95c5-7f402050c97b None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.626153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.626985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.627360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.627703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.628419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:16.628815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.630303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.641752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 371/754] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:16.642796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.656445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.657095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.657377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.657653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.658245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.660495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:16.671088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:16.672072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:16.672902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 384/755] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services/image => generated 86 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:16.678244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.698333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.733311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.771554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae605d7-f7c6-4dac-a750-8fc839ccd35e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.803402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445b154-1827-4e11-a458-26931451aec2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.818590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.819798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.820751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.821343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.822341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] RBAC: Adding query filter params (interface=admin, service_id=3abf1bd590ce4a0899f35bed4910ce79, region_id=RegionOne) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:16.823672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.825687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.833328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 372/756] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/endpoints?service_id=3abf1bd590ce4a0899f35bed4910ce79&interface=admin®ion_id=RegionOne => generated 193 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:16.839722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.847416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:16.848030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:16.848305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:16.848572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:16.849196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] RBAC: Adding query filter params (name=image) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:16.849497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:16.851063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:16.858891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 385/757] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:16.863167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:16.954506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:16.978235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a3059-cb84-4920-b290-c880fbf52e9f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:16.980638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20853ed1-5d86-4fbf-bbb6-f77a0e877595 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:17.018710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:17.019355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.019634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.019904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.020498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:17.022122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:17.023300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:17.024132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.024602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.025109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.026004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] RBAC: Adding query filter params (type=image) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:17.026511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:17.028154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:17.033920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 386/758] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:17.038865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 373/759] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:16 2022] GET /identity/v3/services => generated 1957 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:01:17.042156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:17.157846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b50092-bc4d-41e0-9fa3-416015887311 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:17.197924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:17.198547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.198844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.199136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.199845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] RBAC: Adding query filter params (interface=public, service_id=99331e74b8cf4c748363db2118100256, region_id=RegionOne) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:17.200161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:17.201670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d0b04-8798-4c20-ab43-117a116450a9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:17.208698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 387/760] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:01:17 2022] GET /identity/v3/endpoints?service_id=99331e74b8cf4c748363db2118100256&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:17.213345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:17.224457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:17.320651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb935ac-51e4-4908-9ad3-6fa092b2ce88 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:17.362064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:17.362954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.363377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.363817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.364652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:17.366176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1f81e-8fcf-4627-8fed-afbcc5ae5ca5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:17.373801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 374/761] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:01:17 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:01:17.456268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c71658-a6de-43b9-ba52-8674d1bf6ef5 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.456550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c71658-a6de-43b9-ba52-8674d1bf6ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.456827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c71658-a6de-43b9-ba52-8674d1bf6ef5 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.458462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 388/762] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:17.465024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.465307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.465582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.486045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:17.487638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:17.563836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:17.573172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 375/763] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:17.847760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c71658-a6de-43b9-ba52-8674d1bf6ef5 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:17.851708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:17.956972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c71658-a6de-43b9-ba52-8674d1bf6ef5 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:17.995801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:17.996784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:17.997289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:17.997868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:17.998652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:18.000230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:18.009329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:18.010355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 26 11:01:18.011227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:18.018780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 389/764] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:18.020346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:18.122648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758be696-8a17-48d3-8b5e-b7b17feb80f7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:18.158811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:18.159432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:18.159707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:18.159978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:18.160601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:18.160898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:18.162257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:18.168194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 376/765] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:18.173958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:18.244097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:18.279789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f10ae-d909-4ae0-9b7b-4e4614ee2d52 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:18.318786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:18.319811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:18.320317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:18.320795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:18.321636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:18.322160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:18.323813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c300c-b516-46d9-b313-5332b97d5923 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:18.334427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 390/766] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Oct 26 11:01:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:18.336045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:18.397807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:18.440634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4e029-d085-472f-98c0-2289e91657f4 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:18.475749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:18.476355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:18.476636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:18.476909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:18.477577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:18.479654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19002b38-ad11-4292-be39-276eb7212d05 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:18.560708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 377/767] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:18.878028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc8d3-aa2b-42ec-99c8-f2e59256a612 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:18.878400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc8d3-aa2b-42ec-99c8-f2e59256a612 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:18.878600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fc8d3-aa2b-42ec-99c8-f2e59256a612 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:18.880215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 391/768] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:18.886232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332270b6-bfb6-48c3-a173-0f36c7effa72 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:18.886517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332270b6-bfb6-48c3-a173-0f36c7effa72 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:18.886791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332270b6-bfb6-48c3-a173-0f36c7effa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:18.906479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-332270b6-bfb6-48c3-a173-0f36c7effa72 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:18.908011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-332270b6-bfb6-48c3-a173-0f36c7effa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:18.990922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 378/769] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:18 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:19.263962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc8d3-aa2b-42ec-99c8-f2e59256a612 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:19.287947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497781b2-50f7-4fbb-aff0-46b60fadd409 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.288296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497781b2-50f7-4fbb-aff0-46b60fadd409 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.288503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497781b2-50f7-4fbb-aff0-46b60fadd409 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.290102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 379/770] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:19.295173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.295456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.295732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.315232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:19.316417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:19.358411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:19.395494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fc8d3-aa2b-42ec-99c8-f2e59256a612 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:19.399078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 380/771] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:19 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:19.435369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:19.440603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:19.441209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.441484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.441760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.442408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:19.444007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:19.446657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING keystone.server.flask.common [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 26 11:01:19.496672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.sql.core [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 11:01:19.496672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 26 11:01:19.496672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [parameters: {'user_id': '3472073bde5c4106924946b814858e7f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 26 11:01:19.496672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81745) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-af2fcacc-dc3a-479a-8ce7-5afc4a500749 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:19.504650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:19.505578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application yield resource Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise self.value Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application session.commit() Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 11:01:19.506855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.flush() Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application compat.raise_( Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 11:01:19.513914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:19.514880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:19.516002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [parameters: {'user_id': '3472073bde5c4106924946b814858e7f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:19.517789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 26 11:01:19.519553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:19.521420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 392/772] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Oct 26 11:01:19 2022] POST /identity/v3/users => generated 160 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:19.522297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:19.607071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a40ef-1b03-435d-a47a-6c36dba898cf None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.607419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a40ef-1b03-435d-a47a-6c36dba898cf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.607579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a40ef-1b03-435d-a47a-6c36dba898cf None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.612224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 393/773] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:19.617207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.617482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.617749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.622600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b04b-1fb8-407d-9f00-508dc3e534e0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:19.636740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:19.638238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:19.667154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:19.668045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.668330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.669023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.678859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:19.680571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:01:19.698228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:19.700703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:19.702941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 381/774] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:01:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:19.705088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:19.737452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:19.739143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 394/775] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:19 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:19.740091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:19.827559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa53e80-1f75-42e8-8115-8b69fbccc4f8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:19.862325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:19.863126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.863506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.863879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.864558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:19.866037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:19.873186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6da57b-8451-4920-8e53-9d99fe663bf7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:19.875881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:19.876858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:19.877698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 382/776] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Oct 26 11:01:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:19.883222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:19.902631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:19.918491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:19.919355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:19.919803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:19.920236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:19.921339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:19.922660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:19.924410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:19.966829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 395/777] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Oct 26 11:01:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:19.994255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:20.001506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354e0c-9e54-49d1-b25c-13517a96efdb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.018468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.042608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.043539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.043902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.044248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.045071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:20.045451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.047095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.058707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 383/778] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:20.060337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.152719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97318b-1e76-4285-9e35-4e80ecf48390 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.184220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef1181-3fe8-4dc9-ad58-0fedaeddb511 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.201160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.201884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.202237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.202852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] PATH_INFO: `/v3/services/image` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.203274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.204637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:20.217604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:20.219008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:20.219867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 396/779] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Oct 26 11:01:20 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:20.223514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.238293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.238950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.239234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.239503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.240201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:20.240508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.241873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.247732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 384/780] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Oct 26 11:01:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:20.255025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.286203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:20.288575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:20.343231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2937f-33e9-4792-8a71-3beb49190d7e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.369008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91bbb9-07cb-410b-8f28-4ac8b8d515de None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.379643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.380255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.380843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.381120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.381749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] RBAC: Adding query filter params (name=image) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:20.382058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.383454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.393180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 397/781] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:20.394437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.405367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.405923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.406202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.406468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.407166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.408524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.492907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a7dd-3dc9-4572-8db9-2fddffd50efb None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.504170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:20.510901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 385/782] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:20.537681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.538316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.538597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.538905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.539659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] RBAC: Adding query filter params (type=image) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:20.539969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.541949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c1d8a-5242-4ed3-8e22-e2bb2d693e83 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.553480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 398/783] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:20.555468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:20.649261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:20.668484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3497b3-fce0-4c1b-a9e4-c65a5f76dd45 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:20.704717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:20.705328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.705610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.705884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.706584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:20.708038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f95ca3-714d-4054-a074-994e9ade1563 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:20.802299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 386/784] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:01:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:01:20.859660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fdcf-4bba-46bf-8562-b1bba9826618 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.860018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fdcf-4bba-46bf-8562-b1bba9826618 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.860211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2fdcf-4bba-46bf-8562-b1bba9826618 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.861790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 399/785] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:20.867439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:20.867754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:20.867997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:20.887547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:20.888834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:20.970636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 387/786] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:21.233724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fdcf-4bba-46bf-8562-b1bba9826618 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:21.337669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2fdcf-4bba-46bf-8562-b1bba9826618 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:21.372013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:21.372633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:21.372911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:21.373178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:21.382954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:21.388200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:21.389320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:21.390359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:21.394370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 400/787] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:01:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:21.395670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:21.463816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:21.497495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ab51-4ee6-4754-911e-e54e26895fe2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:21.532835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:21.533441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:21.533720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:21.533990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:21.534646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:21.534980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:21.536339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:21.549043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 388/788] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Oct 26 11:01:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:21.550478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:21.697004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:21.777779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643f719-3b8c-4e04-b5ee-453ef41c8046 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:21.812391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:21.813021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:21.813300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:21.813599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:21.823649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:21.825092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:21.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:21.840464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:21.841390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:21.841390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:21.841390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:21.843169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 401/789] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:21.846401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:21.947306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d9646-784c-41cf-b8c4-70cf8a9f5306 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:21.972837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:21.981774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:21.983353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:21.983353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:21.983353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:21.983572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:21.983910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:21.985314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:21.997608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 389/790] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:22.003066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.084555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:22.102396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd9c78-3725-44cf-99f4-71251f73289d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.136603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.137237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.137519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.137794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.145034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.146455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:01:22.158166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:22.159122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:22.160114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 402/791] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:22.164976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.266621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab8bdb-a33a-43af-a231-306c8458d443 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.302969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.303577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.303858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.304346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.305075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:22.305387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.306849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.328766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 390/792] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:22.329978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.429719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652669e2-f51a-4352-83b5-c9de842f84c8 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.462829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.464092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.464372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.464641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.474431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.475952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.477618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:22.491959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:22.498694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:22.499878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:22.499878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:22.499878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:22.499878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 403/793] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:22.501220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.551262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:22.608184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63742a65-0581-47a3-8bab-18a87ecb8067 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.646140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.646795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.647104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.647376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.648008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:22.648312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.649664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b450ce-0dac-4205-a9aa-b6994852d5de None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.666432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 391/794] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:22.667728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.720048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc0d01-538a-4506-91fd-1c8c7ddd2ea4 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.723356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc0d01-538a-4506-91fd-1c8c7ddd2ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.723356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc0d01-538a-4506-91fd-1c8c7ddd2ea4 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.723356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 392/795] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:22.727589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.727878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.728159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.746286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:22.748481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:22.781941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5effc3-80ac-4cea-9a8f-895e2eacb524 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.816520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.817144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.817420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.817686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.819435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 393/796] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:22.825224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.826646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-7110c377-c6a5-48d0-8811-6d8c10299a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:01:22.837480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:22.838371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:22.844440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 404/797] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:22.845010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:22.879243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:22.950520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134ffad-aded-4d44-9047-c6f180032bf4 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:22.983513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:22.984043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:22.984317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:22.984580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:22.985253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:22.985556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:22.987039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:22.999094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 394/798] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:23.003193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176548bd-a553-4231-a12f-b193e4eb5231 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.003536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176548bd-a553-4231-a12f-b193e4eb5231 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.003825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176548bd-a553-4231-a12f-b193e4eb5231 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.004812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.005265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 405/799] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:23.010225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.011043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.011320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.029549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:23.030744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:23.084288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:23.101223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 406/800] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:23.102062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.106340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:23.112120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e9ca7-5020-43e0-8cbc-94d72e7ca0a6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.148574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.149230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.149510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.149800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/376b0ac2bb5046a48b58293c4a41f632` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.151986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=03aad2f885bc4dc183e934469676a42f, role_id=376b0ac2bb5046a48b58293c4a41f632)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.153442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.201617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cbbbc-62eb-4c19-9447-17137d89e765 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.215168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 395/801] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:01:23 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/03aad2f885bc4dc183e934469676a42f/roles/376b0ac2bb5046a48b58293c4a41f632 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:01:23.238877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.239454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.239730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.239998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.241173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.242659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.246868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 407/802] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:23.251000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.367843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c795b5d-51f1-4371-b334-44671e76edb7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.374133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.405344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.405953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.406226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.406495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.407168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.408583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.429969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:23.474490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d32c3-dac5-4647-82d6-e122e65c0df2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.508440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.508973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.509247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.509513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.510231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.514919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.515440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 396/803] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:23 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:23.561132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.sql.core [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:23.561132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 11:01:23.561132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [parameters: {'id': '9c6c024da77a4fd08c4b9c24cb845de2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 11:01:23.561132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81745) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:23.568211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 26 11:01:23.573182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application yield resource Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application next(self.gen) Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.rollback() Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.force_reraise() Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise self.value Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application session.commit() Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 26 11:01:23.574152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._prepare_impl() Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.session.flush() Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._flush(objects) Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application compat.raise_( Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application flush_context.execute() Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rec.execute(self) Oct 26 11:01:23.575081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 26 11:01:23.576064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result = self._query(query) Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application conn.query(q) Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application result.read() Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 26 11:01:23.576868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [parameters: {'id': '9c6c024da77a4fd08c4b9c24cb845de2', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 26 11:01:23.580134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:23.582293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:23.585534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 408/804] 10.222.0.42 () {60 vars in 1137 bytes} [Wed Oct 26 11:01:23 2022] POST /identity/v3/domains => generated 190 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:23.587331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.591972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:23.692734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312a601-fb94-4a12-918e-fd9ca521b68e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.729993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.730628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.730931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.731203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.741206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.742712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-f06fd416-50e2-4804-8ec3-15ba41a344ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:23.757687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:23.758898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:23.760007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:23.760007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:23.760007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:23.760007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 397/805] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:23.765910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:23.867832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078428b-6a70-43ae-8b86-4b63512c814e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:23.885527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:23.903685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:23.904298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:23.904579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:23.904851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:23.905493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:23.905798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:23.907185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414e42d-3af6-4820-9e7d-e14257a9ada0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:23.921411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 409/806] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:25.353621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58e4f-8546-40de-bd89-b0e64d7d4ad5 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:25.354760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58e4f-8546-40de-bd89-b0e64d7d4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:25.354760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd58e4f-8546-40de-bd89-b0e64d7d4ad5 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:25.356268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 398/807] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:25.362560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b83efc-6858-4208-a9c6-386d7249e554 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:25.363010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b83efc-6858-4208-a9c6-386d7249e554 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:25.363130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b83efc-6858-4208-a9c6-386d7249e554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:25.380690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-83b83efc-6858-4208-a9c6-386d7249e554 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:25.381853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-83b83efc-6858-4208-a9c6-386d7249e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:25.445212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 410/808] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:25.711515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd58e4f-8546-40de-bd89-b0e64d7d4ad5 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:25.716727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f942c8-a223-4911-b9f7-71b3c1264b64 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:25.717093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f942c8-a223-4911-b9f7-71b3c1264b64 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:25.717363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f942c8-a223-4911-b9f7-71b3c1264b64 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:25.718700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 411/809] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:01:25.724058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:25.724344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:25.724629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:25.742433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:25.743673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:25.772335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:25.810568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 412/810] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:25.815163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd58e4f-8546-40de-bd89-b0e64d7d4ad5 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:25.848084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:25.848710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:25.848989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:25.849260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:25.859089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:25.864264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:25.865370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:25.866358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 399/811] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Oct 26 11:01:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:25.871749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:25.905328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:25.967693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc306bb-ebb3-4475-b6fb-1dd7c1e20ab2 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.000153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.000805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.001087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.001357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.002014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:26.002316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.003700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.011937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 413/812] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Oct 26 11:01:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:26.017834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.076065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.114990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29346db-a01b-4f97-9fbd-acd3313482ba None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.134313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.148584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.149213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.149494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.149770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.160301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.161745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.170740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.174474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf433a5a-be39-4a82-b20e-31dbc8c59ec4 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:26.176363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:26.177377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:26.178287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:26.178287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:26.178287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:26.178552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 400/813] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:26.184634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.211052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.211687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.211967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.212246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.213380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.214918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.216487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 414/814] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:26.222183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.282696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3eff8-4f54-4450-9aec-480ffed69b34 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.317196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.317790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.318032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89fa46-b904-4dcd-9785-4cb1ca75dac6 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.318129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.318330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.318995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:26.319299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.320654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.331939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 401/815] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:26.337365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.351068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.351669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.351948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.352219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.352859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.354754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.368111 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.443134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 415/816] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:01:26 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:01:26.443710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7ada9-a652-4fd3-95bd-79a34905b241 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.472025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.472720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.472949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.473217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.481833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.483354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:01:26.496761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:26.497851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:26.498874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 402/817] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:01:26.503939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.530794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.608921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa0582a-9ce2-4bd3-b872-ffee5a4c6a9d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.643343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.643951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.644223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.644491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.645244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:26.645543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.647000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.664323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 416/818] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:26.670761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:26.765092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.773688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ac732-e0e0-4107-8401-39b83c953e71 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:26.807463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:26.808057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:26.808332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:26.808602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:26.873005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:26.874444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:26.959541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:26.969045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 403/819] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:01:26.974705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:27.052087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:27.069840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c1726-bd7e-4c24-9566-2efb4c1c412d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:27.103132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:27.103752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:27.104043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:27.104324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:27.113890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:27.115653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:01:27.129522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:27.130611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:01:27.131540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:01:27.131540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:01:27.131540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:27.131540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 417/820] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:27.136745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:27.233564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905518c-c4db-4687-94cc-555035e6ab26 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:27.266159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:27.267040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:27.267318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:27.267587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:27.268217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:27.268518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:27.269866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:27.281385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 404/821] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:27.286801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:27.384040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79afc4a-080f-45dd-bfce-9d1f25991139 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:27.411896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:27.416707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:27.419672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:27.419672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:27.419672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:27.424472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:27.425889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:01:27.437231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:27.438074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:27.438999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 418/822] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:27.444510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:27.495179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:27.540270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05627edb-ffc4-4c51-bfef-bb9d3d5adc36 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:27.572712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:27.573856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:27.574134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:27.574397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:27.575101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:27.575402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:27.576816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447248-6c73-43b9-b637-332cf40ce432 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:27.588481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 405/823] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:01:27.594580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:27.691474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7982d7-1b54-4785-ad1f-fafb2a641f80 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:27.723865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:27.724479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:27.724761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:27.725035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:27.725762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] RBAC: Adding query filter params (role.id=376b0ac2bb5046a48b58293c4a41f632, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=03aad2f885bc4dc183e934469676a42f) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:27.726067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:27.727528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4a6e-9a21-4b63-8b1f-54aca31dd938 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:27.733935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 419/824] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:01:27 2022] GET /identity/v3/role_assignments?user.id=03aad2f885bc4dc183e934469676a42f&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=376b0ac2bb5046a48b58293c4a41f632 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:28.621674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcd7ea-873c-4b47-a8e4-4541ff671024 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:28.622293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcd7ea-873c-4b47-a8e4-4541ff671024 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:28.622293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcd7ea-873c-4b47-a8e4-4541ff671024 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:28.623612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 406/825] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:28.628732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:28.629169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:28.629559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:28.648168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:28.649380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:28.697697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:28.714120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 420/826] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:28.979278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcd7ea-873c-4b47-a8e4-4541ff671024 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:29.077298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcd7ea-873c-4b47-a8e4-4541ff671024 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:29.089772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:29.109450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:29.110057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:29.110337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:29.110612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:29.121094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:29.126338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:29.127516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:29.128456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 407/827] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:29.133402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:29.228185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd085c0-2275-4491-b031-30e54435d9db None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:29.260663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:29.261203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:29.261478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:29.261748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:29.262393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:29.262694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:29.264072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:29.271934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 421/828] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:29.277427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:29.374240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362472a-117a-4c0b-b67f-c54b1bd2fccc None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:29.406667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:29.407303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:29.407583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:29.407855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] PATH_INFO: `/v3/users/be3dcafadf5b484d8b75a86ea338a8a8` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:29.420839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] RBAC: Authorizing `identity:get_user(user_id=be3dcafadf5b484d8b75a86ea338a8a8)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:29.422269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:29.423854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 408/829] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:29 2022] GET /identity/v3/users/be3dcafadf5b484d8b75a86ea338a8a8 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:29.429364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:29.475394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:29.523391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086a627-05e4-4b84-8ced-60db5054a727 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:29.554621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:29.555254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:29.555532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:29.555803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:29.556903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] RBAC: Authorizing `identity:get_project(project_id=4482fc48e3d14e72b01c67207cba7100)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:29.558935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bddbec-9834-4f69-ba05-e93f63751b59 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:29.565411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 422/830] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:29 2022] GET /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:29.566270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:29.595598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:29.661953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b75466-41be-4b27-b795-98036101c747 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:29.694342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:29.695007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:29.695288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:29.695562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100/users/be3dcafadf5b484d8b75a86ea338a8a8/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:29.697423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] RBAC: Authorizing `identity:create_grant(project_id=4482fc48e3d14e72b01c67207cba7100, user_id=be3dcafadf5b484d8b75a86ea338a8a8, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:29.698849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0034b-a1f0-4379-a061-1aadd2bbdfbd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:29.784219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 409/831] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:01:29 2022] PUT /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100/users/be3dcafadf5b484d8b75a86ea338a8a8/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:01:31.879360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81a36-5aab-4929-b4e5-1a12a81d8793 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:31.879895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81a36-5aab-4929-b4e5-1a12a81d8793 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:31.879895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e81a36-5aab-4929-b4e5-1a12a81d8793 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:31.881235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 423/832] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:31.887198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:31.887598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:31.887744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:31.905617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:31.906810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:31.970573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 410/833] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:32.235926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e81a36-5aab-4929-b4e5-1a12a81d8793 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:32.333334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e81a36-5aab-4929-b4e5-1a12a81d8793 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:32.348596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:32.365479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:32.368747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:32.368747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:32.368747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:32.377026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:32.381753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:32.382869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:32.383848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 424/834] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:32.388704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:32.409040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:32.484717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427b6ab-f449-4b34-9939-b851f4a9634e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:32.517450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:32.517982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:32.518262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:32.518533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:32.519210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:32.519514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:32.520870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:32.528486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 411/835] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:32.534247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:32.629844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34680f-f679-429a-976a-4ecaa95836cc None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:32.662246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:32.662882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:32.663164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:32.663435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] PATH_INFO: `/v3/users/be3dcafadf5b484d8b75a86ea338a8a8` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:32.664538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] RBAC: Authorizing `identity:get_user(user_id=be3dcafadf5b484d8b75a86ea338a8a8)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:32.665944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:32.668146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 425/836] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:32 2022] GET /identity/v3/users/be3dcafadf5b484d8b75a86ea338a8a8 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:32.673793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:32.752205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:32.769579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6f77d-3859-4277-a69c-7d56f7adceb6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:32.802363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:32.802983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:32.803264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:32.803537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:32.867138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:32.868575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:33.058437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:33.077054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 412/837] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:01:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:01:33.083099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:33.128511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:33.180181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de93a26-d1fa-48f5-8d18-876505132a03 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:33.213482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:33.214095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:33.214372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:33.214642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:33.215765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=4482fc48e3d14e72b01c67207cba7100)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:33.217175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c206bc-1a38-4156-8647-f911b437e5d1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:33.218776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 426/838] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:33 2022] GET /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:33.224844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:33.319475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a1892-0a02-4733-a657-b11698db0434 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:33.350989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:33.351571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:33.351847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:33.352118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:33.352829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=4482fc48e3d14e72b01c67207cba7100, user.id=be3dcafadf5b484d8b75a86ea338a8a8) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:33.353134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:33.355139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3284d11c-80e2-41ba-b8eb-59bc718ce4d2 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:33.360569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 413/839] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:01:33 2022] GET /identity/v3/role_assignments?user.id=be3dcafadf5b484d8b75a86ea338a8a8&scope.project.id=4482fc48e3d14e72b01c67207cba7100&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:35.468477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbbfc-39a3-4d94-b1f4-9ed9a6ad01cb None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:35.469042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbbfc-39a3-4d94-b1f4-9ed9a6ad01cb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:35.469042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efbbfc-39a3-4d94-b1f4-9ed9a6ad01cb None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:35.470329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 427/840] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:35.475651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:35.476035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:35.476198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:35.494145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:35.495970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:35.535419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:35.559133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 414/841] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:35.830579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efbbfc-39a3-4d94-b1f4-9ed9a6ad01cb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:35.927035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efbbfc-39a3-4d94-b1f4-9ed9a6ad01cb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:35.939434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:35.960029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:35.960638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:35.960911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:35.961178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:35.962263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:35.963704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035ea32-6bce-40af-874a-9d155b91b8a1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:35.965816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 428/842] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:35.971362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:36.064440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9ec4c-e5c1-4be6-abc5-0c582da09cd1 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:36.096873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:36.097454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:36.097730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:36.098001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:36.098624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:36.100072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3de91-1840-4fa3-a132-7ccde47982fb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:36.161275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 415/843] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:01:35 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:38.283810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08457e35-c081-46f1-a7b4-fd48a16b34ed None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:38.284377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08457e35-c081-46f1-a7b4-fd48a16b34ed None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:38.284377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08457e35-c081-46f1-a7b4-fd48a16b34ed None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:38.285689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 429/844] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:38.290701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:38.291151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:38.291287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:38.309809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:38.311000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:38.335316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:38.375203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 416/845] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:38.640138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08457e35-c081-46f1-a7b4-fd48a16b34ed None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:38.739481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08457e35-c081-46f1-a7b4-fd48a16b34ed None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:38.771805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:38.772419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:38.772698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:38.772969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] PATH_INFO: `/v3/roles/2af5dd46546147a59f096049e368abab` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:38.774318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] RBAC: Authorizing `identity:get_role(role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:38.775696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:38.777516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 430/846] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:38 2022] GET /identity/v3/roles/2af5dd46546147a59f096049e368abab => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:38.783258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:38.876809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41f3bb-9f8a-4835-880c-ceb9f0164d50 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:38.909179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:38.909729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:38.910003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:38.910271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] PATH_INFO: `/v3/users/05c7c37f9140405bb9ec6db6d78a8f6c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:38.922987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] RBAC: Authorizing `identity:get_user(user_id=05c7c37f9140405bb9ec6db6d78a8f6c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:38.924399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:38.925926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 417/847] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:38 2022] GET /identity/v3/users/05c7c37f9140405bb9ec6db6d78a8f6c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:38.931569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:38.978724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:39.027587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98860-da70-44ba-9ac7-21787d293cf7 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:39.058990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:39.059595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:39.059873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:39.060145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:39.061259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] RBAC: Authorizing `identity:get_project(project_id=4482fc48e3d14e72b01c67207cba7100)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:39.062672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123aad6-f6cc-4097-9f9d-4899431d7b27 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:39.064290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 431/848] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:38 2022] GET /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:39.069930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:39.091641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:39.166645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc38d85-f046-48ba-9699-6582f524c700 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:39.198014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:39.199157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:39.199430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:39.199696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100/users/05c7c37f9140405bb9ec6db6d78a8f6c/roles/2af5dd46546147a59f096049e368abab` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:39.201752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] RBAC: Authorizing `identity:create_grant(project_id=4482fc48e3d14e72b01c67207cba7100, user_id=05c7c37f9140405bb9ec6db6d78a8f6c, role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:39.203174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e9d4d9-1210-4f25-9a78-aaa2970e537d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:39.270400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 418/849] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:01:39 2022] PUT /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100/users/05c7c37f9140405bb9ec6db6d78a8f6c/roles/2af5dd46546147a59f096049e368abab => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:01:41.360606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122564e0-234e-44da-802c-2c06b652c8da None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:41.361132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122564e0-234e-44da-802c-2c06b652c8da None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:41.361132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122564e0-234e-44da-802c-2c06b652c8da None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:41.362494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 432/850] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:41.367552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:41.368000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:41.368095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:41.385523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:41.387232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:41.451062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 419/851] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:41.719609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122564e0-234e-44da-802c-2c06b652c8da None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:41.816736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122564e0-234e-44da-802c-2c06b652c8da None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:41.832629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:41.848999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:41.849614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:41.849891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:41.850162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] PATH_INFO: `/v3/roles/2af5dd46546147a59f096049e368abab` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:41.851564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] RBAC: Authorizing `identity:get_role(role_id=2af5dd46546147a59f096049e368abab)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:41.852913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:41.854214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 433/852] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:41 2022] GET /identity/v3/roles/2af5dd46546147a59f096049e368abab => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:41.859943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:41.871607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:41.955964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b534f-3c99-42ee-9f76-aacaa571df6d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:41.988171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:41.988731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:41.989009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:41.989280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] PATH_INFO: `/v3/users/05c7c37f9140405bb9ec6db6d78a8f6c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:41.990373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] RBAC: Authorizing `identity:get_user(user_id=05c7c37f9140405bb9ec6db6d78a8f6c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:41.991803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:41.993345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 420/853] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:41 2022] GET /identity/v3/users/05c7c37f9140405bb9ec6db6d78a8f6c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:41.999487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:42.092792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1733b03c-3188-42e9-abf9-f6b8cc0e13b3 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:42.125418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:42.125994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:42.126268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:42.126534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:42.174937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:42.189865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:42.191350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:42.283824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 434/854] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:01:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:01:42.289491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:42.365357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:42.385681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da98b5-3a2e-4b4a-918d-11252ece6afd None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:42.419403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:42.420009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:42.420288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:42.420560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] PATH_INFO: `/v3/projects/4482fc48e3d14e72b01c67207cba7100` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:42.421655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] RBAC: Authorizing `identity:get_project(project_id=4482fc48e3d14e72b01c67207cba7100)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:42.423084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6648ce-7453-4e7f-9e25-b11e3aece0fa None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:42.424678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 421/855] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:42 2022] GET /identity/v3/projects/4482fc48e3d14e72b01c67207cba7100 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:42.430728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:42.509028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:42.524087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa803e73-1cc7-4d48-a524-c11fbd92ac7e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:42.557017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:42.557552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:42.557875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:42.558150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:42.558891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] RBAC: Adding query filter params (role.id=2af5dd46546147a59f096049e368abab, scope.project.id=4482fc48e3d14e72b01c67207cba7100, user.id=05c7c37f9140405bb9ec6db6d78a8f6c) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:42.559201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:42.560607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18892fbc-2558-47be-8a14-66e9db59fa97 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:42.566066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 435/856] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:01:42 2022] GET /identity/v3/role_assignments?user.id=05c7c37f9140405bb9ec6db6d78a8f6c&scope.project.id=4482fc48e3d14e72b01c67207cba7100&role.id=2af5dd46546147a59f096049e368abab => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:44.696086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf184-0674-4600-b739-5aaec8b68c19 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:44.696614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf184-0674-4600-b739-5aaec8b68c19 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:44.696614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf184-0674-4600-b739-5aaec8b68c19 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:44.697960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 422/857] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:44.703089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:44.703527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:44.703620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:44.721651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:44.722852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:44.786286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 436/858] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:45.052408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf184-0674-4600-b739-5aaec8b68c19 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:45.149023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf184-0674-4600-b739-5aaec8b68c19 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:45.181099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:45.181703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:45.181981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:45.182252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:45.183352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:45.184758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03d99-ab1e-4e72-a83a-5344b1e2ec6f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:45.186280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 423/859] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:45.192062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:45.265296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:45.285550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0385f-1b97-456e-bbdd-d5b0421bf1b0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:45.318136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:45.318667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:45.318973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:45.319246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:45.319891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:45.321294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58a146-8f17-4617-a273-edd578019766 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:45.391963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 437/860] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:01:45 2022] POST /identity/v3/projects => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:47.559669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9c54-ea38-4579-a829-dc791c883b61 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:47.560246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9c54-ea38-4579-a829-dc791c883b61 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:47.560246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f9c54-ea38-4579-a829-dc791c883b61 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:47.561579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 424/861] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:47.566954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:47.567368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:47.567502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:47.585202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:47.586377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:47.649263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 438/862] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:47.915325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f9c54-ea38-4579-a829-dc791c883b61 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:47.931052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:48.013201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f9c54-ea38-4579-a829-dc791c883b61 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:48.045949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:48.046577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:48.046879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:48.047156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:48.048275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:48.049695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83150c2a-5a93-41c1-9e10-68efe575c96c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:48.051275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 425/863] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:01:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:48.056992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:48.113144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:48.151324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ed1cc-af58-4cc8-a66d-ff98e994c00b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:48.184240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:48.184781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:48.185061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:48.185333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:48.186009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:48.187526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc5173-0369-4f32-9a14-26824cc56127 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:48.296492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 439/864] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:01:48 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:01:50.457521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb39788-b820-42aa-b729-b8ddf9d6bf31 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.458081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb39788-b820-42aa-b729-b8ddf9d6bf31 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.458081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb39788-b820-42aa-b729-b8ddf9d6bf31 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.459435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 426/865] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:50.464514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.464940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.465072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.484001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:50.485175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:50.547996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 440/866] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:01:50.819440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb39788-b820-42aa-b729-b8ddf9d6bf31 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:50.899741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:50.918655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb39788-b820-42aa-b729-b8ddf9d6bf31 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:50.951023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:50.951635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:50.951914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:50.952187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:50.961793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:50.967142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:50.968206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:01:50.969213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 427/867] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:50.974174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:51.018038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:51.068581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f04e2a-775b-4b2f-86a5-27adf9eb4894 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:51.101078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:51.101664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:51.101943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:51.102213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:51.102890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:51.103199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:51.104547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:51.112186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 441/868] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:51.117786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:51.212984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a645c-5034-461c-b0bc-0ac70f3b8e95 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:51.245061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:51.245677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:51.245953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:51.246221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] PATH_INFO: `/v3/users/a12c2cf4621c48999fcd7f42512ecc85` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:51.256532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:51.258778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] RBAC: Authorizing `identity:get_user(user_id=a12c2cf4621c48999fcd7f42512ecc85)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:51.260207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:51.262264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 428/869] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:51 2022] GET /identity/v3/users/a12c2cf4621c48999fcd7f42512ecc85 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:51.268324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:51.364978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212dd8-2c8a-4dbf-aafe-ac655c6e1cd0 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:51.373885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:51.398508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:51.399154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:51.399436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:51.399707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] PATH_INFO: `/v3/projects/1008122478e7464fac4db2d48fd05989` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:51.400812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] RBAC: Authorizing `identity:get_project(project_id=1008122478e7464fac4db2d48fd05989)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:51.402255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52eff6-f16c-4254-b750-e96a83f8fd43 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:51.403885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 442/870] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:51 2022] GET /identity/v3/projects/1008122478e7464fac4db2d48fd05989 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:51.409643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:51.504029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b3d88-79d7-492f-95b6-904ffa86b543 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:51.536515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:51.537089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:51.537364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:51.537633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] PATH_INFO: `/v3/projects/1008122478e7464fac4db2d48fd05989/users/a12c2cf4621c48999fcd7f42512ecc85/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:51.539464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1008122478e7464fac4db2d48fd05989, user_id=a12c2cf4621c48999fcd7f42512ecc85, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:51.540858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da9e0d-cc94-4072-be92-415581047fa1 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:51.674554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 429/871] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:01:51 2022] PUT /identity/v3/projects/1008122478e7464fac4db2d48fd05989/users/a12c2cf4621c48999fcd7f42512ecc85/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:01:53.768856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee922e0-9673-4a9b-b6da-b0c2dfb07490 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:53.769264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee922e0-9673-4a9b-b6da-b0c2dfb07490 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:53.769378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee922e0-9673-4a9b-b6da-b0c2dfb07490 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:53.770721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 443/872] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:53.776029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:53.776407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:53.776564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:53.795110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:53.796269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:53.848394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:53.859507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 430/873] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:54.127159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee922e0-9673-4a9b-b6da-b0c2dfb07490 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:54.225219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee922e0-9673-4a9b-b6da-b0c2dfb07490 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.257932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.258542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.258850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.259128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.269081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:01:54.273812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:01:54.274934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:01:54.275858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 444/874] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:01:54.280729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:54.375122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b049f8-5db5-4cce-beb4-eec625c6cb19 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.407817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.408343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.408619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.408889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.409527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:54.409826 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:54.411193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:54.419540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 431/875] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:54.425322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:54.458977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:54.520735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f106c-83e2-42d5-9d32-f005ff46395d None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.553591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.554201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.554480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.554753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] PATH_INFO: `/v3/users/a12c2cf4621c48999fcd7f42512ecc85` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.555897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=a12c2cf4621c48999fcd7f42512ecc85)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:54.557305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:54.558920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 445/876] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/users/a12c2cf4621c48999fcd7f42512ecc85 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:54.564564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:54.615886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:54.660982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a53921-73df-4a47-bf47-3d1b1863c464 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.693378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.693953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.694226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.694490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.760176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:54.762105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:54.810353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:54.855987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 432/877] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:01:54.861821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:54.957130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e902bd-7f76-47e5-8523-67a63aec6bd2 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:54.959297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:54.988941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:54.989558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:54.989835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:54.990105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] PATH_INFO: `/v3/projects/1008122478e7464fac4db2d48fd05989` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:54.991223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] RBAC: Authorizing `identity:get_project(project_id=1008122478e7464fac4db2d48fd05989)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:54.992633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9df20a-de74-410a-973b-6ee134c65584 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:54.994228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 446/878] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/projects/1008122478e7464fac4db2d48fd05989 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:55.000311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:55.095091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0cae-6df6-478c-93d4-3b899f2bd26d None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:55.127233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:55.127812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:55.128088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:55.128357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:55.129053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=1008122478e7464fac4db2d48fd05989, user.id=a12c2cf4621c48999fcd7f42512ecc85) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:01:55.129355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:55.130751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00742d8-ca26-422d-a6b5-572affb2bc85 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:55.136810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 433/879] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:01:54 2022] GET /identity/v3/role_assignments?user.id=a12c2cf4621c48999fcd7f42512ecc85&scope.project.id=1008122478e7464fac4db2d48fd05989&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:01:57.264668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728157-7054-4dcb-bfcf-d5535ab20369 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.265230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728157-7054-4dcb-bfcf-d5535ab20369 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.265230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728157-7054-4dcb-bfcf-d5535ab20369 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.266546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 447/880] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:01:57.272145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159617f-1e24-41b1-817c-5bdc12bc7009 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.272540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159617f-1e24-41b1-817c-5bdc12bc7009 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.272690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159617f-1e24-41b1-817c-5bdc12bc7009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.290967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e159617f-1e24-41b1-817c-5bdc12bc7009 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:01:57.292146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e159617f-1e24-41b1-817c-5bdc12bc7009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:01:57.302415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:01:57.356471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 434/881] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:01:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:01:57.623055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728157-7054-4dcb-bfcf-d5535ab20369 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:01:57.719110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728157-7054-4dcb-bfcf-d5535ab20369 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:01:57.752615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:01:57.753216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:01:57.753497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:01:57.753771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:01:57.754445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:01:57.755847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b91901-b84e-424e-b72f-b93846723c44 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:01:57.829304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 448/882] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Oct 26 11:01:57 2022] POST /identity/v3/domains => generated 267 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:02:00.039527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f42b4-90c7-44af-b78a-647b9c24bebd None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:00.040076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f42b4-90c7-44af-b78a-647b9c24bebd None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:00.040076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f42b4-90c7-44af-b78a-647b9c24bebd None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:00.041333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 435/883] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:00.046560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:00.047020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:00.047140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:00.064799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:00.065979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:00.091205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:00.129199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 449/884] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:02:00.396027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f42b4-90c7-44af-b78a-647b9c24bebd None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:00.493725 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f42b4-90c7-44af-b78a-647b9c24bebd None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:00.525882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:00.526488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:00.527281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:00.527550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] PATH_INFO: `/v3/domains/6a5457981ba046098efdc900f02e1a9e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:00.537404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a5457981ba046098efdc900f02e1a9e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:00.538840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91839f65-0e89-4792-ba24-d5c282a3212b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:00.540421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 436/885] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Oct 26 11:02:00 2022] GET /identity/v3/domains/6a5457981ba046098efdc900f02e1a9e => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:00.546696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:00.640825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c75081-94e5-446c-81e1-8903bb191f1a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:00.673960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:00.674504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:00.674785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:00.675092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:00.675748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:00.677174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d1e97-0da3-4026-9c35-15bd3a60a4e9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:00.767270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 450/886] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:02:00 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:02:02.947684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319eba1-8eab-4fa7-8dd1-995ec40adee2 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:02.948218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319eba1-8eab-4fa7-8dd1-995ec40adee2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:02.948218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319eba1-8eab-4fa7-8dd1-995ec40adee2 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:02.949612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 437/887] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:02.954870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:02.955284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:02.955426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:02.973136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:02.974314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:02.983734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:03.038426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 451/888] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:02:03.306610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319eba1-8eab-4fa7-8dd1-995ec40adee2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:03.316305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:03.405049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319eba1-8eab-4fa7-8dd1-995ec40adee2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:03.436438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:03.437040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:03.437315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:03.437582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] PATH_INFO: `/v3/domains/6a5457981ba046098efdc900f02e1a9e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:03.438660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a5457981ba046098efdc900f02e1a9e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:03.440078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888e90d-be11-4673-a80c-483298c66b86 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:03.441597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 438/889] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Oct 26 11:02:03 2022] GET /identity/v3/domains/6a5457981ba046098efdc900f02e1a9e => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:03.447237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:03.539910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00347f6-d519-43fc-b078-9090f9ab3655 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:03.571843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:03.572368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:03.572644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:03.572944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:03.574163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:03.575599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0690a7-940f-498b-8362-0ad971251066 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:03.655692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:03.660402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 452/890] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:02:03 2022] POST /identity/v3/users => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:02:05.804520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce1fc-605f-44b8-b1f0-7ad9f4c3c3d9 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:05.804983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce1fc-605f-44b8-b1f0-7ad9f4c3c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:05.805080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ce1fc-605f-44b8-b1f0-7ad9f4c3c3d9 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:05.806391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 439/891] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:05.811629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:05.812037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:05.812190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:05.830568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:05.831790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:05.895007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 453/892] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:02:06.161047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ce1fc-605f-44b8-b1f0-7ad9f4c3c3d9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:06.233976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:06.258263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ce1fc-605f-44b8-b1f0-7ad9f4c3c3d9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:06.291344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:06.291949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:06.292224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:06.292511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:06.302446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:02:06.307120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:02:06.308759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:02:06.309715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 440/893] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:02:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:02:06.314611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:06.409231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e6cff-e5ea-43dc-898a-eab70976657b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:06.436730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:06.442291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:06.446615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:06.454128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 454/894] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:02:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:06.459717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:06.553905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86fb70-35d9-4c4c-9ed7-ee541004506f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:06.586381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:06.587013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:06.587290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:06.587560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] PATH_INFO: `/v3/users/85dd1f5a8a3b41d99b49bdb388557c80` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:06.589354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:06.669362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] RBAC: Authorizing `identity:get_user(user_id=85dd1f5a8a3b41d99b49bdb388557c80)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:06.670810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:06.672387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 441/895] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:02:06 2022] GET /identity/v3/users/85dd1f5a8a3b41d99b49bdb388557c80 => generated 311 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:06.678262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:06.774831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdfe5a-3ccd-4063-a46b-6d7d2223a5ba None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:06.806758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:06.807399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:06.807675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:06.807943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] PATH_INFO: `/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:06.809620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] RBAC: Authorizing `identity:get_project(project_id=5354ff92ba5c46f38ce485cbf0cfcce8)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:06.811063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fee01-7a1d-41fd-8494-16be697c1c23 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:06.812668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 455/896] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:02:06 2022] GET /identity/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:06.818672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:06.913618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d319a39-adce-4666-9c0f-80f305677748 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:06.946494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:06.947150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:06.947430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:06.947705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] PATH_INFO: `/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8/users/85dd1f5a8a3b41d99b49bdb388557c80/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:06.949585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] RBAC: Authorizing `identity:create_grant(project_id=5354ff92ba5c46f38ce485cbf0cfcce8, user_id=85dd1f5a8a3b41d99b49bdb388557c80, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:06.951015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c7511-aec2-4788-9a79-ecc188ec5043 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:07.022803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 442/897] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:02:06 2022] PUT /identity/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8/users/85dd1f5a8a3b41d99b49bdb388557c80/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:02:09.133033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ecc98-2054-48ca-920c-78b6ce6dc35b None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:09.133532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ecc98-2054-48ca-920c-78b6ce6dc35b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:09.133623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ecc98-2054-48ca-920c-78b6ce6dc35b None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:09.134943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 456/898] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:02:09.140274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:09.140716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:09.140816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:09.158640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:02:09.159853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:02:09.205230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:09.223758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 443/899] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:02:09.496229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ecc98-2054-48ca-920c-78b6ce6dc35b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:09.553597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:09.593114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ecc98-2054-48ca-920c-78b6ce6dc35b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:09.625132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:09.625745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:09.626025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:09.626297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:09.636505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:02:09.641218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:02:09.642315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:02:09.643649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 457/900] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:02:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:02:09.648451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:09.743658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d972235-c8d4-4a2e-b60d-b98b1860db93 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:09.778183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:09.778708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:09.779013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:09.779287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:09.779931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:02:09.780233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:09.781581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:09.789260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 444/901] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:02:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:09.795221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:09.890321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4a73a-680a-47a1-9e14-2e7e372a3472 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:09.909515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:09.922529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:09.923164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:09.923444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:09.923714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] PATH_INFO: `/v3/users/85dd1f5a8a3b41d99b49bdb388557c80` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:09.924818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=85dd1f5a8a3b41d99b49bdb388557c80)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:09.926228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:09.927832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 458/902] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Oct 26 11:02:09 2022] GET /identity/v3/users/85dd1f5a8a3b41d99b49bdb388557c80 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:09.933604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:09.977873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:10.030231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda8318-a4b0-4d2e-95b3-8d81a027f0e5 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:10.062368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:10.062991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:10.063278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:10.063547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:10.127787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:10.129203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:10.171117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:10.223542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 445/903] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:02:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:02:10.229453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:10.324899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea49f4-95b4-48cc-bb4b-1aa8b4232ae3 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:10.355869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:10.356471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:10.356757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:10.357029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] PATH_INFO: `/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:10.358131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] RBAC: Authorizing `identity:get_project(project_id=5354ff92ba5c46f38ce485cbf0cfcce8)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:10.359562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1839be-1e9f-4464-b7a4-6c995604a4fd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:10.361129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 459/904] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Oct 26 11:02:10 2022] GET /identity/v3/projects/5354ff92ba5c46f38ce485cbf0cfcce8 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:10.367100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:02:10.462560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d03446-e08f-4d0d-bac8-cf6c002dc733 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:02:10.495361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:02:10.496544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:02:10.496823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:02:10.497094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:02:10.497801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=5354ff92ba5c46f38ce485cbf0cfcce8, user.id=85dd1f5a8a3b41d99b49bdb388557c80) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:02:10.498106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:02:10.499531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a5fe0c-2c83-493e-8dd7-ab6ad6fb5696 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:02:10.505105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 446/905] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:02:10 2022] GET /identity/v3/role_assignments?user.id=85dd1f5a8a3b41d99b49bdb388557c80&scope.project.id=5354ff92ba5c46f38ce485cbf0cfcce8&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:02:20.351907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=72 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:02:20.502597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:22.345080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11749c71-e046-4059-ab5e-fa46bc2be939 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:22.345669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11749c71-e046-4059-ab5e-fa46bc2be939 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:22.345669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11749c71-e046-4059-ab5e-fa46bc2be939 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:22.347042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 460/906] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:22.352769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:22.359061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:22.359061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:22.371271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:22.377284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:22.437349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 447/907] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 26 11:03:22.718715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11749c71-e046-4059-ab5e-fa46bc2be939 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:22.820641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11749c71-e046-4059-ab5e-fa46bc2be939 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:22.852791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:22.853417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:22.853699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:22.853976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:22.863787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:22.865217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:22.880032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:22.881260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:22.882257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:22.882257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:22.882257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:22.882257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 461/908] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:22.887524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:22.983399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed0700-c8a2-45ba-a8f3-ff48393baa79 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:23.011995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:23.017911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:23.023856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cc9b7-08f8-486a-9326-5a5fba91b990 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:23.032398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 448/909] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:23.038086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:23.089751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:23.135835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662d20b-822b-45bd-bdc7-df308e6577ed None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:23.167260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:23.167846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:23.168571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:23.168848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:23.169478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:23.170906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0ff0e-8702-4dc2-b1fd-c221b479802a None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:23.260831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 462/910] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Oct 26 11:03:23 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:03:25.385700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb338b54-b2d3-4c74-a7af-c9654cd9fe72 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:25.386262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb338b54-b2d3-4c74-a7af-c9654cd9fe72 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:25.386262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb338b54-b2d3-4c74-a7af-c9654cd9fe72 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:25.387604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 449/911] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:25.392883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:25.393283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:25.393460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:25.412724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:25.413949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:25.478424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 463/912] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:25.743617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb338b54-b2d3-4c74-a7af-c9654cd9fe72 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:25.844693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb338b54-b2d3-4c74-a7af-c9654cd9fe72 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:25.876926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:25.877547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:25.877827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:25.878101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:25.888862 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:03:25.893969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:25.895753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:25.896795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 450/913] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:03:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:25.901921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:25.939806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:25.996014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d409-b314-4a37-87f7-a318c92cf2f7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.028574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.029170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.029449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.029719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.030371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] RBAC: Adding query filter params (name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:26.030671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.032068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.040277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 464/914] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:03:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:26.046050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.094632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:26.146251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e58a0-d5bb-4533-ac01-775d23faa7ac None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.178219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.178840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.179158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.179428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.189244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.190659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:26.204936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:26.206047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:26.207138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:26.207138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:26.207138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:26.207138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 451/915] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:26.211908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.306761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:26.309517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdb763-ec2a-46e8-b88a-6cb604b22467 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.341677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.342287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.342549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.342844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.343481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:26.343786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.345134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.356875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 465/916] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:26.362699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.463128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef88a4-a171-4052-b08f-075985c9bd4c None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.464654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:26.496120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.496732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.497009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.497278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.504852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.506218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:03:26.516992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:26.518376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:26.519427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 452/917] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:26.524747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.620271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ddda0-dbdd-40ab-b906-dc8d4576b42f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.653905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.654505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.654779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.655075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.655826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:26.656116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.657524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.673897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 466/918] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:26.680165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.777208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14120cc6-7cf9-4705-8a82-ee3f10f84135 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.809981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.810598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.810910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.811184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.821348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.822769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:26.839343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:26.840469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:26.841831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:26.841831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:26.841831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:26.841831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 453/919] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:26.846263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:26.866423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:26.943717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28062011-aa0e-45db-9ad8-6546f6396a80 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:26.977847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:26.978477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:26.978755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:26.979131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:26.980135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:26.980443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:26.981788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:26.992800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 467/920] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:26.998624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:27.015614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:27.095278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55f560-2688-4ac1-9fe6-bbfde5a9b864 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:27.128543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:27.129848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:27.130137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:27.130403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:27.136820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:27.138232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:03:27.149841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:27.150954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:27.152099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 454/921] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:27.156805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:27.233540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:27.254222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f2add-f41b-4038-9449-a0f03f86da33 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:27.286652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:27.287282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:27.287605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:27.287876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:27.288550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:27.288853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:27.290264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c40a5-7a2e-4797-b6de-18796e32470e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:27.302235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 468/922] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:03:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:27.308341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:27.381675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:27.406325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da25be-096c-4ae6-8b2c-f985ea0abdfb None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:27.440937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:27.442126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:27.442404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:27.442672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e4a26f6a0712462996a17c7fe00a68ef/roles/5623d92ae8b2487cb36f2849eeed41c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:27.456744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=e4a26f6a0712462996a17c7fe00a68ef, role_id=5623d92ae8b2487cb36f2849eeed41c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:27.458181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386b155-bea3-486c-a2b1-b12768a2a7c8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:27.525251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 455/923] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:03:27 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e4a26f6a0712462996a17c7fe00a68ef/roles/5623d92ae8b2487cb36f2849eeed41c6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:03:29.660682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a390d6-0786-4f70-a7b3-94f5d5107845 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:29.661261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a390d6-0786-4f70-a7b3-94f5d5107845 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:29.661261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a390d6-0786-4f70-a7b3-94f5d5107845 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:29.662676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 469/924] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:29.667797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:29.668226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:29.668382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:29.686610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:29.687828 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:29.752490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 456/925] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:30.022981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a390d6-0786-4f70-a7b3-94f5d5107845 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.121822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a390d6-0786-4f70-a7b3-94f5d5107845 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.153954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.154572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.154873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.155144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.162175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:30.165918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:03:30.170593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:30.171721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:30.172989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 470/926] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:30.178239 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.231073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:30.273520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e94fd2-6bae-4842-95e3-c857ec3e5f9c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.306242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.306855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.307140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.307406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.308052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] RBAC: Adding query filter params (name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:30.308354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:30.309707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:30.317674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 457/927] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:30.323613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.422290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df54ee-9490-46e8-bc4c-d016131fe92f None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.455357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.455975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.456258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.457121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.466628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:30.468083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:30.482582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:30.483752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:30.485037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:30.485037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:30.485037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:30.485037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 471/928] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:30.489667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.588305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c142997-412a-4f85-b33a-b9ffdc3b6279 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.604446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:30.621358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.621905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.622179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.622450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.623108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:30.623407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:30.624761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:30.636573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 458/929] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:03:30.642607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.692122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:30.739670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96389fc6-5310-49f7-b4bd-af3e1f329cfa None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.772536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.773156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.773432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.773702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.780833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:30.782362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:03:30.793763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:30.795172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:30.796466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 472/930] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:03:30.800709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:30.902347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff901df2-f5b3-484b-a9ed-9b8261bff7e8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:30.936401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:30.937002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:30.937279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:30.937549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:30.938261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:30.938687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:30.940146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:30.958912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 459/931] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:30.964701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.061120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a970d5-bb1c-455f-89ab-806b7a6f6609 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:31.071474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:31.094027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:31.094617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:31.094919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:31.095191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:31.160269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:31.161795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:31.255160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 473/932] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:03:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:03:31.261479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.295387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:31.360389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ade86-90fa-4f37-9660-749ce09d8036 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:31.394316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:31.394969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:31.395252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:31.395547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:31.405198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:31.406619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:31.421097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:31.422377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:31.423700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:31.423700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:31.423700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:31.423700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 460/933] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:31.428424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.439755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:31.524978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386012f-9d4e-4956-9ea8-ee026b99f7d1 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:31.558174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84483198-234e-4350-8059-61e9f3742237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:31.558762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84483198-234e-4350-8059-61e9f3742237 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:31.559066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84483198-234e-4350-8059-61e9f3742237 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:31.559336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84483198-234e-4350-8059-61e9f3742237 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:31.559961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84483198-234e-4350-8059-61e9f3742237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:31.560264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84483198-234e-4350-8059-61e9f3742237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:31.561613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84483198-234e-4350-8059-61e9f3742237 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:31.572282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 474/934] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:31.578049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.660815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:31.675446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08dbef-f6c7-4fc2-acd4-7bda14d3fb6b None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:31.708200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:31.708830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:31.709310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:31.709985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:31.716562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:31.717985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:03:31.729456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:31.736308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:31.737405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 461/935] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:31.738007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84483198-234e-4350-8059-61e9f3742237 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.803845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:31.833138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84483198-234e-4350-8059-61e9f3742237 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:31.865306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:31.866504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:31.866894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:31.867116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:31.867787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:31.868088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:31.869490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37b7a0-5962-4d62-97a0-da91eb3b5107 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:31.881068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 475/936] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:31.888797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:31.985623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98e5c2-29df-48f9-b893-8672cf0cbcf3 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:32.019757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:32.020356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:32.020630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:32.020899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:32.021597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] RBAC: Adding query filter params (role.id=5623d92ae8b2487cb36f2849eeed41c6, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=e4a26f6a0712462996a17c7fe00a68ef) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:32.022459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:32.023837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188ac7a-ca85-4f5e-8509-b19e757f26aa None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:32.029395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 462/937] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:03:31 2022] GET /identity/v3/role_assignments?user.id=e4a26f6a0712462996a17c7fe00a68ef&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=5623d92ae8b2487cb36f2849eeed41c6 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:34.204862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72fd54-99b6-4436-8506-e4e367887cd7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:34.205426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72fd54-99b6-4436-8506-e4e367887cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:34.205426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f72fd54-99b6-4436-8506-e4e367887cd7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:34.206761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 476/938] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:34.212573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:34.212989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:34.213113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:34.231315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:34.232494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:34.235202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:34.296217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 463/939] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:34.560780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72fd54-99b6-4436-8506-e4e367887cd7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:34.660399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f72fd54-99b6-4436-8506-e4e367887cd7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:34.687453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:34.693085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:34.694198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:34.694472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:34.694739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:34.704351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:03:34.709055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:34.713391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:34.714305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 477/940] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:03:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:34.716778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:34.813950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a1083-dbd6-4ea6-bea6-a794839cbc5e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:34.847207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:34.847818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:34.848076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:34.848343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:34.848986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:34.849286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:34.850662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:34.862394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 464/941] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:34.868751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:34.965236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69c9de-bdcd-4810-a130-7a57fd850a9a None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:34.997997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:34.998728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:34.999031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:34.999302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.009829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.011310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:35.026065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:35.027151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:35.028490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:35.028490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:35.028490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:35.028490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 478/942] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:35.033106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.035944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.130603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e23ae-fb0b-423a-99d8-fbf85f369e9a None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.164123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.164709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.164984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.165252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.165869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:35.166169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.167532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.179592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 465/943] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:35.185392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.227189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.281319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88abfe9-b21a-4108-9f07-2a1ac37ba193 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.313887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.314500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.314776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.315076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.322226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.323666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:03:35.334771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:35.335933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:35.336967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:35.338981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 479/944] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 26 11:03:35.341791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.401332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.439620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed11a68-bb54-4be6-8d21-4135784f10d7 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.472471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.473113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.473395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.473661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.474371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:35.474675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.476597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.493637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 466/945] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:35.499602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.595499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935fe95-ac25-48b2-a3a8-7462f34298a7 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.596956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.628742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.629360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.629652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.629906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.639873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.641272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:35.656294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:35.657359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:35.658324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:35.658324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:35.658324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:35.658866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 480/946] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:35.663976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.761800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107767b-1c33-460a-8a7f-04c778f1b1e2 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.783525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:35.797285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.797875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.798191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.798459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.799116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:35.799418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.800788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.811919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 467/947] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:35.817675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:35.915276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb7d3d-8ec4-49a0-90df-6b0479e160bd None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:35.947243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:35.947865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:35.948146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:35.948423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:35.954832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:35.956287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:03:35.967729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:35.968755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:35.969861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 481/948] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:35.975490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:36.072098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8e951-063a-4002-b163-0b48c0bd387e None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:36.104033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:36.104618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:36.104892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:36.105161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:36.105830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:36.106129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:36.107627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f3dc6-9a8a-4929-b6cd-82c9a1faf8a5 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:36.119051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 468/949] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:03:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:36.125297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:36.141680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:36.223165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8880b346-c6a6-4bf9-b658-15cd08c03186 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:36.255255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:36.255898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:36.256672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:36.256945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] PATH_INFO: `/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e4a26f6a0712462996a17c7fe00a68ef/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:36.258846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=398b5b4391a3473484fc6044da2bc128, user_id=e4a26f6a0712462996a17c7fe00a68ef, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:36.260261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443fc1a-b092-4145-b036-fabae6afa1c9 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:36.340568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 482/950] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Oct 26 11:03:36 2022] PUT /identity/v3/projects/398b5b4391a3473484fc6044da2bc128/users/e4a26f6a0712462996a17c7fe00a68ef/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:03:38.452454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3aa0e-b3e5-43d5-a911-658bd1b326d5 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:38.453066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3aa0e-b3e5-43d5-a911-658bd1b326d5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:38.453066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3aa0e-b3e5-43d5-a911-658bd1b326d5 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:38.454469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 469/951] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:38.459780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:38.460193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:38.460358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:38.479121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:38.480306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:38.543928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 483/952] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:38.812050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3aa0e-b3e5-43d5-a911-658bd1b326d5 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:38.854397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:38.910783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3aa0e-b3e5-43d5-a911-658bd1b326d5 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:38.944524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:38.945118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:38.945395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:38.945662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:38.955499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 26 11:03:38.960727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:38.961891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:38.962790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 470/953] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Oct 26 11:03:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:38.967942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:38.971555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:39.062077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca1307-2ab4-43d1-bd2f-513de1e408d8 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.094208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.094858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.095133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.095405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.096070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:39.096372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.097742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.106041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 484/954] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Oct 26 11:03:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:03:39.111955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:39.210017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:39.212849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e58858-55b6-4e7a-b4f8-d7176652ffd8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.244536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.245151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.245432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.245704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.255809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.257734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:39.271461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:39.273157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:39.274069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:39.274069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:39.274069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:39.274069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 471/955] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:39.279024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:39.342984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:39.376376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a123a-0983-4f1e-b0ce-11cd75b1daac None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.408843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.409439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.409812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.410088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.410723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:39.411065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.412438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.423626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 485/956] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:39.429906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:39.527561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defef399-0c50-44cc-8fcc-81493298c5a8 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.560521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.561134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.561413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.561686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.569528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.570980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 26 11:03:39.581882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:39.583052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:39.584574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 472/957] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:39.588806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:39.685370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe004-1f56-4b5c-b21a-cd4e0c51ea32 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.710571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:39.719039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.719635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.719909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.720205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.720929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:39.721227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.722687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.741130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 486/958] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Oct 26 11:03:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:39.750082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:39.762672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:39.843466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c607016-df68-4f1a-a828-bcc3a33f6b70 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:39.876879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:39.877482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:39.877762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:39.878035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:39.943723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:39.945097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:39.957908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:40.038404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 473/959] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Oct 26 11:03:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 26 11:03:40.044470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:40.138352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42647f48-4235-486e-816f-2e6c485f4dfb None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:40.171823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:40.172440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:40.172719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:40.172991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:40.182510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:40.183969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 26 11:03:40.198636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:40.199741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 26 11:03:40.200916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 26 11:03:40.200916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 26 11:03:40.200916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:40.200916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 487/960] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:40.205491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:40.302162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2732b-4cfb-4237-8517-40270e6bf618 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:40.314995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:40.335629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:40.336222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:40.336501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:40.336773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:40.337400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:40.337704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:40.339080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:40.350737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 474/961] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Oct 26 11:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:40.356542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:40.390137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:40.451107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11c18f-a44b-4c0c-b885-faff2c53b6ea None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:40.484668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:40.485283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:40.485563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:40.485839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:40.492237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:40.493729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 26 11:03:40.504697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:40.505736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: ERROR keystone.server.flask.application Oct 26 11:03:40.507363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 488/962] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Oct 26 11:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:40.512234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:40.610489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8208223-d2d7-4e9c-b2d2-32fa5d828ef6 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:40.643937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:40.644530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:40.644808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:40.645077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:40.645750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:40.646051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:40.648021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6dd18-6be9-429f-a269-5fa49d204687 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:40.659682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 475/963] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Oct 26 11:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:40.665764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:40.758356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:40.763940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15520e05-8718-4b02-ab62-9a76279afe10 None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:40.796324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:40.796913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:40.797192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:40.797461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:40.798169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] RBAC: Adding query filter params (role.id=27a106926ca14908b7e268013b8452a2, scope.project.id=398b5b4391a3473484fc6044da2bc128, user.id=e4a26f6a0712462996a17c7fe00a68ef) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:40.798475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:40.799922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624314-0e10-40d9-9a43-351568b7c099 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:40.805517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 489/964] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Oct 26 11:03:40 2022] GET /identity/v3/role_assignments?user.id=e4a26f6a0712462996a17c7fe00a68ef&scope.project.id=398b5b4391a3473484fc6044da2bc128&role.id=27a106926ca14908b7e268013b8452a2 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:42.989450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47b69d-59ea-445a-99bf-8194b233aef2 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.989995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47b69d-59ea-445a-99bf-8194b233aef2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.989995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47b69d-59ea-445a-99bf-8194b233aef2 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:42.991947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 476/965] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:42.997415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:42.997846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:42.997952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:43.015563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:43.016731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:43.081564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 490/966] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:43.352063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47b69d-59ea-445a-99bf-8194b233aef2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:43.382328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:43.449702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47b69d-59ea-445a-99bf-8194b233aef2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:43.483124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:43.483741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:43.484251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:43.484566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] PATH_INFO: `/v3/services/placement` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:43.485364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:43.487337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:43.496041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:43.497305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:43.498470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 477/967] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:03:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:43.503233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:43.590838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:43.600289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7387c-e99b-4b1b-b809-a9fe1e1c3a97 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:43.632420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:43.633037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:43.633315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:43.633624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:43.634258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:43.634556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:43.635939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c907e-9a51-49cb-9799-cbe9db061792 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:43.640696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 491/968] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Oct 26 11:03:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:03:43.647411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:43.739146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:43.744902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c42700-231e-4a09-afbc-14d1fd7cd6df None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:43.778573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:43.779212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:43.779490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:43.779760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] PATH_INFO: `/v3/services` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:43.780379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:43.780684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:43.782036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a7f4-33cc-4a37-ac35-c6ecd2c26430 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:43.786274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 478/969] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Oct 26 11:03:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:03:45.914954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574fee-f559-4366-a099-b98c3cdca8c8 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:45.915507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574fee-f559-4366-a099-b98c3cdca8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:45.915507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77574fee-f559-4366-a099-b98c3cdca8c8 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:45.916859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 492/970] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:45.921907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7914-dedf-4e1b-8c12-8c1cc4cdbeb4 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:45.922311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7914-dedf-4e1b-8c12-8c1cc4cdbeb4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:45.922475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d7914-dedf-4e1b-8c12-8c1cc4cdbeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:45.940730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-474d7914-dedf-4e1b-8c12-8c1cc4cdbeb4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:45.941900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-474d7914-dedf-4e1b-8c12-8c1cc4cdbeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:46.006279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 479/971] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:46.277117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77574fee-f559-4366-a099-b98c3cdca8c8 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:46.375402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77574fee-f559-4366-a099-b98c3cdca8c8 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:46.408726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:46.409311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:46.409587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:46.409854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:46.410485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:46.412470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f41c7a-801c-4ee0-846d-28084b61bbba None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:46.508191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 493/972] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Oct 26 11:03:46 2022] POST /identity/v3/services => generated 246 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:03:48.620000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4e07-7dbe-4281-89f0-8a92638ecddb None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:48.620553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4e07-7dbe-4281-89f0-8a92638ecddb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:48.620553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4e07-7dbe-4281-89f0-8a92638ecddb None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:48.621884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 480/973] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:48.627131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:48.627571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:48.627710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:48.646354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:48.650001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:48.656588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:48.718853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 494/974] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:48.981320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4e07-7dbe-4281-89f0-8a92638ecddb None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:49.072525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:49.082730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4e07-7dbe-4281-89f0-8a92638ecddb None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:49.115781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:49.116401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.116680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.116953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.117523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:49.118897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:49.127386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:49.128577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:49.129636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 481/975] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:03:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:49.134924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:49.241107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72306f42-be29-4566-bd3e-da4c4d8f4a58 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:49.276510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:49.277131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.277405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.277676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.278311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:49.278612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:49.280025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:49.285859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 495/976] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Oct 26 11:03:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:49.292991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:49.405422 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00aac2-4a9b-4f96-b851-8f8e6953605f None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:49.441309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:49.441932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.442216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.442494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.443303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] RBAC: Adding query filter params (interface=public, service_id=2b01919c4a8f4669807b5330836930e3, region_id=RegionOne) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:49.443616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:49.444994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23278053-2e08-4e37-82e1-df5e45401595 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:49.450683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 482/977] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Oct 26 11:03:49 2022] GET /identity/v3/endpoints?service_id=2b01919c4a8f4669807b5330836930e3&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:03:49.456559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:49.462037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:49.563356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031823f-edcf-4749-a31b-3eb98964de8e None admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:49.598504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:49.599140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:49.599422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:49.599696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:49.600296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:49.601666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbbde1-9458-48b4-8112-d44daa5f0946 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:49.608724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 496/978] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Oct 26 11:03:49 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:03:51.726994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba2054-0c30-4a40-b066-6d2f3b182607 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:51.727520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba2054-0c30-4a40-b066-6d2f3b182607 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:51.727520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba2054-0c30-4a40-b066-6d2f3b182607 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:51.728927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 483/979] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:03:51.734032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:51.734413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:51.734578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:51.753309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:03:51.754531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:03:51.814718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:51.820047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 497/980] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:03:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 26 11:03:52.088491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba2054-0c30-4a40-b066-6d2f3b182607 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:52.108603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:52.189865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba2054-0c30-4a40-b066-6d2f3b182607 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:52.222173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:52.222780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:52.223092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:52.223361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:52.223989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:52.225856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 26 11:03:52.234358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 26 11:03:52.236181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.get_or_create( Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application with Lock( Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self._enter() Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application return self.creator() Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application created_value = creator( Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: ERROR keystone.server.flask.application Oct 26 11:03:52.237224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 484/981] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Oct 26 11:03:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 26 11:03:52.241729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:52.335346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0d18-6a44-44ac-a066-d962ad749e9d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:52.367331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:52.367962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:52.368244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:52.368517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] PATH_INFO: `/v3/services` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:52.369144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:03:52.369448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:52.370812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206ec47-3c8e-410e-a7d1-cbbf70134845 None admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:52.375875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 498/982] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Oct 26 11:03:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:03:52.381587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:03:52.461326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:03:52.479212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58f7c-df2f-4153-85fe-50f48e320ab3 None admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:03:52.512345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:03:52.512931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:03:52.513208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:03:52.513479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:03:52.514118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:03:52.515490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068cf08d-42ca-4c9f-b4d7-eea1411c9d10 None admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:03:52.605531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 485/983] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:03:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:04:02.372875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=69 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:04:02.600981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:50.424093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be0d7-dd24-46bf-b3ac-04739d317904 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:50.424680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be0d7-dd24-46bf-b3ac-04739d317904 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:50.424680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380be0d7-dd24-46bf-b3ac-04739d317904 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:50.427246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 499/984] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:09:50.431299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979706d7-9fd4-4592-bd60-a37dff860e4c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:50.431741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979706d7-9fd4-4592-bd60-a37dff860e4c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:50.431836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979706d7-9fd4-4592-bd60-a37dff860e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:50.544223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-979706d7-9fd4-4592-bd60-a37dff860e4c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:09:50.544711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-979706d7-9fd4-4592-bd60-a37dff860e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:50.757565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 486/985] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:09:51.065805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:51.065998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:51.066269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:51.067927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 500/986] 10.222.0.42 () {54 vars in 964 bytes} [Wed Oct 26 11:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:09:51.074053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f446-e06f-4fa4-be95-f4074427b795 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:51.074344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f446-e06f-4fa4-be95-f4074427b795 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:51.074623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67f446-e06f-4fa4-be95-f4074427b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:51.176007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-2e67f446-e06f-4fa4-be95-f4074427b795 None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:09:51.177410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2e67f446-e06f-4fa4-be95-f4074427b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:51.202414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:51.268908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 487/987] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Oct 26 11:09:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:09:51.278612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:09:51.280821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:51.424984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100cf2e-af53-437d-994e-323cb8da2a16 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:09:51.508278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:09:51.509710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:51.509710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:51.509710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:51.511112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:51.573849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:51.640766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:09:51.642206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7fe97-cef4-4508-b74d-ba369b864e48 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:09:51.666393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 501/988] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:09:54.138282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:54.138771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:54.138896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:54.140901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 488/989] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:09:54.145952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c43989-e596-4fd3-8567-14c338597fdf None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:54.146378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c43989-e596-4fd3-8567-14c338597fdf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:54.146501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c43989-e596-4fd3-8567-14c338597fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:54.228607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f0c43989-e596-4fd3-8567-14c338597fdf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:09:54.229820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f0c43989-e596-4fd3-8567-14c338597fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:54.330847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 502/990] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:09:54.610880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:09:54.612407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:54.713686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e84c0-ed18-4df7-855a-48b750e7cb18 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:09:54.787733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:54.790034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:09:54.790640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:54.790948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:54.791217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:54.792789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:54.899729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:09:54.901150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70728f5b-b374-4b0c-9697-e6ac9baf47ab service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:09:54.919123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 489/991] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:09:57.566798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:57.567366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:57.567366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:57.568725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 503/992] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:09:57.574010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b75f-0759-4e72-ab09-893b4db125a3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:57.574403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b75f-0759-4e72-ab09-893b4db125a3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:57.574560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2b75f-0759-4e72-ab09-893b4db125a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:57.638922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-91a2b75f-0759-4e72-ab09-893b4db125a3 None None] MFA Rules not processed for user `a996e65d4f0b46038e59c86823161af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:09:57.640131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-91a2b75f-0759-4e72-ab09-893b4db125a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:57.691017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:57.735037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 490/993] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:09:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:09:58.021158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:09:58.022698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:58.032398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:58.122859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666f216-83d1-438e-9e20-90d45b4b418c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:09:58.194727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:09:58.195351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:09:58.195631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:09:58.195902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:09:58.197500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:09:58.258073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:09:58.302899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:09:58.304263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426abab1-2409-4e30-a434-8ee7366f7645 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:09:58.322986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 504/994] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:09:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:01.213612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fbf98-86ff-4b9b-ad94-ac8812266fa4 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:01.214179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fbf98-86ff-4b9b-ad94-ac8812266fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:01.214179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fbf98-86ff-4b9b-ad94-ac8812266fa4 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:01.215578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 491/995] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:01.221544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6355-4b23-4e9e-9d3c-e637669521fa None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:01.221943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6355-4b23-4e9e-9d3c-e637669521fa None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:01.222095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc6355-4b23-4e9e-9d3c-e637669521fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:01.287002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-08fc6355-4b23-4e9e-9d3c-e637669521fa None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:01.288203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-08fc6355-4b23-4e9e-9d3c-e637669521fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:01.381817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 505/996] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:01.684705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:01.684989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:01.685264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:01.686657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 492/997] 10.222.0.42 () {54 vars in 964 bytes} [Wed Oct 26 11:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:01.692323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ac9ce-7db5-4ff2-b23f-34977a2e879e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:01.692616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ac9ce-7db5-4ff2-b23f-34977a2e879e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:01.692895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ac9ce-7db5-4ff2-b23f-34977a2e879e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:01.716115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:01.770913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0f5ac9ce-7db5-4ff2-b23f-34977a2e879e None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:01.772085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0f5ac9ce-7db5-4ff2-b23f-34977a2e879e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:01.845693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 506/998] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Oct 26 11:10:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:01.854610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:01.856178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:01.957405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf11bae-e84a-4c56-88d5-4a8f13430e7e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:02.030688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:02.031660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:02.031971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:02.032246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:02.033806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:02.040537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:02.141635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:02.143085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0debe17-c88f-4c57-a620-e8128a5ceef2 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:02.161201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 493/999] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:05.637629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:05.638209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:05.638209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:05.639577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 507/1000] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:05.645124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc0bd-e427-457f-b7ae-68c48b560a6f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:05.645603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc0bd-e427-457f-b7ae-68c48b560a6f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:05.645695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc0bd-e427-457f-b7ae-68c48b560a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:05.714320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-779bc0bd-e427-457f-b7ae-68c48b560a6f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:05.715568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-779bc0bd-e427-457f-b7ae-68c48b560a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:05.777467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:05.822340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 494/1001] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:06.107797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:06.109317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:06.212571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbd163-7c75-45e5-8fd1-06ec680277fa None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:06.224734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:06.285814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:06.286416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:06.286695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:06.287002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:06.288613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:06.397358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:06.398791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd74b2e-a233-4656-8e73-dff9e51dcff0 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:06.416359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 508/1002] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:08.618965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:08.619504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:08.619504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:08.620886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 495/1003] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:08.626273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ded6a2-16b4-4015-837a-632ea014e3fb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:08.626742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ded6a2-16b4-4015-837a-632ea014e3fb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:08.627031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ded6a2-16b4-4015-837a-632ea014e3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:08.680952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:08.698887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-93ded6a2-16b4-4015-837a-632ea014e3fb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:08.700137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-93ded6a2-16b4-4015-837a-632ea014e3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:08.795752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 509/1004] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:09.077414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:09.078976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:09.183757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5bb10-b2aa-4983-8943-2ef793202218 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:09.258245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:09.258870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:09.259153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:09.259432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:09.260986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:09.278993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:09.369924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:09.371383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8efa5-eb66-4ccc-bae2-5e53496bb97e service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:09.390119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 496/1005] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:11.598790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:11.599370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:11.599370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:11.600724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 510/1006] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:11.606117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdda49a-a467-4e3d-8805-6085ba262bc7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:11.606530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdda49a-a467-4e3d-8805-6085ba262bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:11.606670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdda49a-a467-4e3d-8805-6085ba262bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:11.672132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0cdda49a-a467-4e3d-8805-6085ba262bc7 None None] MFA Rules not processed for user `a996e65d4f0b46038e59c86823161af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:11.673260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0cdda49a-a467-4e3d-8805-6085ba262bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:11.737364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:11.775099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 497/1007] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:12.056505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:12.058054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:12.165102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56bc55-5ed6-47da-90f8-a81573255d6b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:12.186507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:12.241419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:12.242031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:12.242310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:12.242887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:12.244472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:12.363254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:12.364696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f476-449d-4ee9-ada7-908022a3fe79 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:12.387734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 511/1008] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:14.983109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:14.983676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:14.983676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:14.984998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 498/1009] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:14.990252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa2f9-1664-44d9-b535-7c14887e1a32 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:14.990678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa2f9-1664-44d9-b535-7c14887e1a32 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:14.990780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffa2f9-1664-44d9-b535-7c14887e1a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:15.053162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:15.055683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-70ffa2f9-1664-44d9-b535-7c14887e1a32 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:15.056887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-70ffa2f9-1664-44d9-b535-7c14887e1a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:15.151894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 512/1010] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:15.431982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:15.433508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:15.535067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf335e0-0461-41c8-b3b1-238e6e549f6c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:15.609257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:15.609849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:15.610125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:15.610393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:15.611983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:15.639070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:15.721724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:15.723131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd17d-671b-4b8d-9d7b-d1ba00033ba4 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:15.740879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 499/1011] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:17.929272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:17.929791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:17.929791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:17.931197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 513/1012] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:17.937013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636256b0-0166-4f67-9942-bbc3b5f619e2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:17.937432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636256b0-0166-4f67-9942-bbc3b5f619e2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:17.937581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636256b0-0166-4f67-9942-bbc3b5f619e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:18.001769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-636256b0-0166-4f67-9942-bbc3b5f619e2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:18.002940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-636256b0-0166-4f67-9942-bbc3b5f619e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:18.074362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:18.095343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 500/1013] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:18.380975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:18.382513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:18.482798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ebf63d-5428-4495-a49a-f5a00e73f4e7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:18.510029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:18.554050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:18.554644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:18.554948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:18.555220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:18.556789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:18.661887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:18.663342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fcce9-d9de-48fe-9fce-85af59441d0b service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:18.681145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 514/1014] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:21.485433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.485993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.485993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.487346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 501/1015] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:21.492802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdeb-a831-4c9e-9e51-b45de0308b5c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:21.493221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdeb-a831-4c9e-9e51-b45de0308b5c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:21.493332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dfdeb-a831-4c9e-9e51-b45de0308b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:21.557715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-834dfdeb-a831-4c9e-9e51-b45de0308b5c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:21.558949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-834dfdeb-a831-4c9e-9e51-b45de0308b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:21.567499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:21.653897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 515/1016] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:21.936784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:21.938313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.039134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a5bb0-11d1-42f9-af90-3335c375ca88 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:22.110968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:22.117089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:22.117089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:22.117089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:22.117089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:22.148000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:22.222411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:22.223860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5d1fa-e8a0-4972-96cc-7bc42cf5c519 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:22.242667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 502/1017] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:26.583170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:26.583701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:26.583701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:26.585055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 516/1018] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:26.590236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c55e9-5408-4a63-8e24-ee73b85f480b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:26.590632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c55e9-5408-4a63-8e24-ee73b85f480b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:26.590778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c55e9-5408-4a63-8e24-ee73b85f480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:26.656532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d03c55e9-5408-4a63-8e24-ee73b85f480b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:26.657733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d03c55e9-5408-4a63-8e24-ee73b85f480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:26.755185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:26.767541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 503/1019] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:27.051278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:27.052802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:27.155805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ca17c-e7ec-4af4-a3eb-32c6c1911e04 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:27.195029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:27.229970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:27.230792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:27.231138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:27.232154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:27.234116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:27.344558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:27.345883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd545f03-2bde-42bc-a2ab-b5908e0c62e7 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:27.364761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 517/1020] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:30.529272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.529829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.529829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.531211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 504/1021] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:30.536731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac791e-41f0-47c9-988a-311237f70f99 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:30.537176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac791e-41f0-47c9-988a-311237f70f99 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:30.537399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac791e-41f0-47c9-988a-311237f70f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:30.603622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ecac791e-41f0-47c9-988a-311237f70f99 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:30.604730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ecac791e-41f0-47c9-988a-311237f70f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:30.622917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:30.699324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 518/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:30.984290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:30.985937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:31.091087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d4c1-4682-41b5-8594-0d8a2422d026 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:31.165818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:31.166452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:31.166738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:31.167025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:31.168675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:31.230624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:31.305182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:31.306727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddde8319-4dc4-4d31-9901-897271563356 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:31.327207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 505/1023] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:35.757198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.757642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.757745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.759180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 519/1024] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:35.765495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5aaf0-a401-4620-bb83-a2116f95cc09 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:35.766023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5aaf0-a401-4620-bb83-a2116f95cc09 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:35.766104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5aaf0-a401-4620-bb83-a2116f95cc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:35.834485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-52f5aaf0-a401-4620-bb83-a2116f95cc09 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:35.835748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-52f5aaf0-a401-4620-bb83-a2116f95cc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:35.932465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:35.935591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 506/1025] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:36.222574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:36.224180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:36.328081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218a0bf-435d-43b3-a14d-9f8db0028b78 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:36.376790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:36.403238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:36.403838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:36.404111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:36.404375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:36.405943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:36.517892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:36.519375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0a043-c17e-4bdb-8b23-5a7667edb6b6 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:36.537567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 520/1026] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:41.416263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff175d-c5a9-4b75-9e72-ffa04cdeeccf None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.416728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff175d-c5a9-4b75-9e72-ffa04cdeeccf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.416817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff175d-c5a9-4b75-9e72-ffa04cdeeccf None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.419077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 507/1027] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:41.424845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356be3-8497-499a-8f9e-1f5a2f66e6b8 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:41.425282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356be3-8497-499a-8f9e-1f5a2f66e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:41.425369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a356be3-8497-499a-8f9e-1f5a2f66e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:41.580569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0a356be3-8497-499a-8f9e-1f5a2f66e6b8 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:41.581803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a356be3-8497-499a-8f9e-1f5a2f66e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:41.609580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:41.712266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 521/1028] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:10:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:45.910652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee469-ba2e-4c11-a30f-5a5892a420b1 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.911559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee469-ba2e-4c11-a30f-5a5892a420b1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.911998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee469-ba2e-4c11-a30f-5a5892a420b1 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:45.913864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 508/1029] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:45.919458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf41bb-6bf7-484c-b830-4ae9a2fa81c6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:45.919875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf41bb-6bf7-484c-b830-4ae9a2fa81c6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:45.920014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf41bb-6bf7-484c-b830-4ae9a2fa81c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:46.004209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3daf41bb-6bf7-484c-b830-4ae9a2fa81c6 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:46.005642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3daf41bb-6bf7-484c-b830-4ae9a2fa81c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:46.098166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:46.140524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 522/1030] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:10:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:47.318229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dba3-8e2f-4b9e-a653-9a9057f69e8f None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.318767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dba3-8e2f-4b9e-a653-9a9057f69e8f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.319108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603dba3-8e2f-4b9e-a653-9a9057f69e8f None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.320868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 509/1031] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:47.326456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16da3a-0707-400b-909c-111020228b74 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:47.326905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16da3a-0707-400b-909c-111020228b74 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:47.327051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16da3a-0707-400b-909c-111020228b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:47.406982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-aa16da3a-0707-400b-909c-111020228b74 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:47.408195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-aa16da3a-0707-400b-909c-111020228b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:47.515164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 523/1032] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:10:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:51.068255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe0145-fa20-4cb4-aed8-28b9fa71760d None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.068756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe0145-fa20-4cb4-aed8-28b9fa71760d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.068756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe0145-fa20-4cb4-aed8-28b9fa71760d None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.070030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 510/1033] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:51.076105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df13f3-58f7-448e-a96e-4d846f34f077 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.076611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df13f3-58f7-448e-a96e-4d846f34f077 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.076956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df13f3-58f7-448e-a96e-4d846f34f077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.115038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:51.147079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-73df13f3-58f7-448e-a96e-4d846f34f077 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:51.148289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-73df13f3-58f7-448e-a96e-4d846f34f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.253901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 524/1034] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:51.557381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.557915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.558397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.560277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 511/1035] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:51.567398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a366-01c4-42b4-b7a1-982e574d445d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:51.569143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a366-01c4-42b4-b7a1-982e574d445d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:51.569143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a366-01c4-42b4-b7a1-982e574d445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:51.658988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0c98a366-01c4-42b4-b7a1-982e574d445d None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:51.660618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0c98a366-01c4-42b4-b7a1-982e574d445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.719387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:51.808984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 525/1036] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:10:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:51.809780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:51.811578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:51.977564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e84f2-a770-4f32-bae9-6352fe6a9393 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:52.054298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:52.096409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:52.097077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:52.097459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:52.097829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:52.099629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:52.204537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:52.206631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac8303-e0dc-463b-8301-abd86e7d4f67 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:52.227262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 512/1037] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:10:54.621008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:54.622063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:54.622063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:54.623164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 526/1038] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:54.630849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bbef-c230-4e2c-94d2-4aec37fb8a38 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:54.631223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bbef-c230-4e2c-94d2-4aec37fb8a38 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:54.632184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90bbef-c230-4e2c-94d2-4aec37fb8a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:54.703245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6b90bbef-c230-4e2c-94d2-4aec37fb8a38 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:54.704495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6b90bbef-c230-4e2c-94d2-4aec37fb8a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:54.748209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:54.810514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 513/1039] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:55.101217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:55.103253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:55.285182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3692356-fb7a-433c-9505-38e2d212715f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:55.296917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:55.418199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:55.418861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:55.419121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:55.419393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:55.420980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:55.516976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:55.518480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abaa93-9e1f-4d02-b9ad-265d5b3f2e00 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:55.537105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 527/1040] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:58.155439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029d75-1fae-4c0d-b46b-eb2e76fa77bd None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.156007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029d75-1fae-4c0d-b46b-eb2e76fa77bd None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.156109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6029d75-1fae-4c0d-b46b-eb2e76fa77bd None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.157670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 514/1041] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:58.163233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c9b6f-ddb6-403b-990d-8b374d7eb369 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.163628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c9b6f-ddb6-403b-990d-8b374d7eb369 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.167608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58c9b6f-ddb6-403b-990d-8b374d7eb369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.180594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:58.235281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c58c9b6f-ddb6-403b-990d-8b374d7eb369 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.236491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c58c9b6f-ddb6-403b-990d-8b374d7eb369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.337927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 528/1042] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.632374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.632925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.633220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.634699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 515/1043] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:58.640275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a7a57-a204-47a5-b092-be4504769771 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:58.640655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a7a57-a204-47a5-b092-be4504769771 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:58.641003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a7a57-a204-47a5-b092-be4504769771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:58.724072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-482a7a57-a204-47a5-b092-be4504769771 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:58.725582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-482a7a57-a204-47a5-b092-be4504769771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:58.764443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:58.851849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 529/1044] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:10:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:58.860403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:58.862056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.018921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3c570-a7b2-40ae-a66f-4e47b7ab51e2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:10:59.040300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:59.143912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:10:59.144519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.144803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.145076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.146657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.200754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95db8-a1a3-4374-8029-b12ab79196aa None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.201313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95db8-a1a3-4374-8029-b12ab79196aa None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.201313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b95db8-a1a3-4374-8029-b12ab79196aa None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.202793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 530/1045] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:10:59.207547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffedef-4641-4e75-8a46-67bcf2ae322a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.207834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffedef-4641-4e75-8a46-67bcf2ae322a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.208109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffedef-4641-4e75-8a46-67bcf2ae322a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.244207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:10:59.245795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f16b4-3bc9-4775-8b5b-57ae190a02d5 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:10:59.265989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 516/1046] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:10:59.278198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-baffedef-4641-4e75-8a46-67bcf2ae322a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:59.279559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-baffedef-4641-4e75-8a46-67bcf2ae322a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.379355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:10:59.388011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 531/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:10:59.720909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.720909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.720909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.722432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 517/1048] 10.222.0.42 () {54 vars in 963 bytes} [Wed Oct 26 11:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:10:59.726865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ac03-7dda-445c-9023-1595710b59ce None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:10:59.727040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ac03-7dda-445c-9023-1595710b59ce None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:10:59.727316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ac03-7dda-445c-9023-1595710b59ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:10:59.820247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6272ac03-7dda-445c-9023-1595710b59ce None None] MFA Rules not processed for user `e3b32b5787164f1ba63e5e1e53f78174`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:10:59.821724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6272ac03-7dda-445c-9023-1595710b59ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.904752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 532/1049] 10.222.0.42 () {58 vars in 1060 bytes} [Wed Oct 26 11:10:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:10:59.912565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:10:59.914164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:10:59.950981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:00.031084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c05f3-d92c-4043-acbb-76da5bc527f3 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:00.108671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:00.109383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:00.109728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:00.110067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:00.111766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:00.200106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:00.232579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:00.233941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451d3a8-b98a-439a-8d17-57bc74f658f1 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:00.263031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 518/1050] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:01.632670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.633212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.633212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.634581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 533/1051] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:01.639752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47e296-e4b9-4db6-97c8-f355809f30a1 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:01.640189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47e296-e4b9-4db6-97c8-f355809f30a1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:01.640444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47e296-e4b9-4db6-97c8-f355809f30a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:01.707078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-dd47e296-e4b9-4db6-97c8-f355809f30a1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:01.708261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dd47e296-e4b9-4db6-97c8-f355809f30a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:01.812888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 519/1052] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:02.105761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:02.107350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:02.194476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:02.258516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832b140-d40d-4ff6-937d-cc39fdffbf1e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:02.379997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:02.380615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:02.380899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:02.381174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:02.382776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:02.433921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:02.475862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:02.477461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7551d72-d8aa-43dc-a47f-f9dd544c2871 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:02.496612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 534/1053] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:04.855350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.855878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.856160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.857669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 520/1054] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:04.863046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6b620-2303-4548-99fa-40c58b24bf7b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:04.863719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6b620-2303-4548-99fa-40c58b24bf7b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:04.864025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6b620-2303-4548-99fa-40c58b24bf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:04.932748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-85f6b620-2303-4548-99fa-40c58b24bf7b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:04.933959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-85f6b620-2303-4548-99fa-40c58b24bf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.036709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 535/1055] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:05.326623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:05.328334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.368079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:05.484310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d487b-9dbc-43b3-b1ed-b303cdbbf816 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:05.599323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:05.605081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:05.605793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:05.606170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:05.606537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:05.608517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:05.704079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:05.705740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e97187-63dc-4c9b-8de1-3c3e719ace17 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:05.735184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 521/1056] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:08.034304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:08.034902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:08.034902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:08.036214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 536/1057] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:08.041518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219a90-0246-4eea-804e-51d8092b4b89 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:08.041949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219a90-0246-4eea-804e-51d8092b4b89 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:08.042052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d219a90-0246-4eea-804e-51d8092b4b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:08.109301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8d219a90-0246-4eea-804e-51d8092b4b89 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:08.110503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8d219a90-0246-4eea-804e-51d8092b4b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.178743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:08.222638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 522/1058] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:08.515157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:08.517230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.553168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:08.673063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3043c62-2eb5-44cb-b67a-1e25f9cc7717 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:08.779758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:08.798647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:08.800235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:08.800235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:08.800235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:08.801670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:08.902480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:08.904109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2fcf9-2d6f-472d-be71-77b74313ff8e service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:08.921905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 537/1059] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:11.202315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:11.202896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:11.202896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:11.204293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 523/1060] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:11.209681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67a9-2dd1-4b97-bda4-18178bc6edd0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:11.210215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67a9-2dd1-4b97-bda4-18178bc6edd0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:11.210215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67a9-2dd1-4b97-bda4-18178bc6edd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:11.276379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-09ea67a9-2dd1-4b97-bda4-18178bc6edd0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:11.277453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-09ea67a9-2dd1-4b97-bda4-18178bc6edd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:11.335152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:11.384913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 538/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:11.669308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:11.671417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:11.834412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200662b-9c8c-4c15-a874-da5c32044391 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:11.897452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:11.981897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:11.982770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:11.983086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:11.983404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:11.985553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:12.098002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:12.099631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d7a01-4ece-4e96-908b-18b04399701b service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:12.117619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 524/1062] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:14.294250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.294881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.294881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.296182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 539/1063] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:14.301698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204258-77ad-4eda-b0a4-8fa0bae471be None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:14.302127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204258-77ad-4eda-b0a4-8fa0bae471be None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:14.302327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204258-77ad-4eda-b0a4-8fa0bae471be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:14.369005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-80204258-77ad-4eda-b0a4-8fa0bae471be None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:14.370223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-80204258-77ad-4eda-b0a4-8fa0bae471be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.378552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:14.468394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 525/1064] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:14.755557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:14.757106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:14.905785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d85c2-d5dc-4f13-8c55-6b64da4cb7f6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:14.950310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:15.015128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:15.015644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.015918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.016189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.017780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.104873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:15.106460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a314d9-4b43-4ab8-9fd4-d9b9436cc5e1 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:15.125123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 540/1065] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:15.752652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0cef0-601a-46a7-9b44-f444e16947eb None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.753119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0cef0-601a-46a7-9b44-f444e16947eb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.753213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0cef0-601a-46a7-9b44-f444e16947eb None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.754683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 526/1066] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:15.759956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adce90-8a77-4277-8640-63a6c1771c2a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:15.760458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adce90-8a77-4277-8640-63a6c1771c2a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:15.760541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adce90-8a77-4277-8640-63a6c1771c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:15.825849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:15.828462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b2adce90-8a77-4277-8640-63a6c1771c2a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:15.829675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b2adce90-8a77-4277-8640-63a6c1771c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:15.926876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 541/1067] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:16.229036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.229317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.229588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.230956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 527/1068] 10.222.0.42 () {54 vars in 962 bytes} [Wed Oct 26 11:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:11:16.236427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b38bb-64f5-49b6-8e31-c6960b6d5693 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.236706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b38bb-64f5-49b6-8e31-c6960b6d5693 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.236972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b38bb-64f5-49b6-8e31-c6960b6d5693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.312801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8d3b38bb-64f5-49b6-8e31-c6960b6d5693 None None] MFA Rules not processed for user `59ac752182c14071999766921f6b4f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:16.313996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8d3b38bb-64f5-49b6-8e31-c6960b6d5693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.382659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:16.388763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 542/1069] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Oct 26 11:11:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:16.396559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:16.398189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.499136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b9007-0864-4cd0-8b91-00531113ce29 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:16.536820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:16.573809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:16.574411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:16.574694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:16.574999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:16.576621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:16.684838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:16.686278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c96fe8-6322-4b31-ab6e-e0d19f395c7e service glance] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:16.704986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 528/1070] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:17.455666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce681c-f5b1-4fc9-a846-52a4b3ddc17d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.456199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce681c-f5b1-4fc9-a846-52a4b3ddc17d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.456199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce681c-f5b1-4fc9-a846-52a4b3ddc17d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.457467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 543/1071] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:17.462810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d216-ae6f-44d2-b1b0-ef71ba5e3a3c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.463266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d216-ae6f-44d2-b1b0-ef71ba5e3a3c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.463376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140d216-ae6f-44d2-b1b0-ef71ba5e3a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.531531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0140d216-ae6f-44d2-b1b0-ef71ba5e3a3c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.532739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0140d216-ae6f-44d2-b1b0-ef71ba5e3a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.550391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:17.618970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c4cb-b51b-4fd1-8501-aea1a032fb55 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.619257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c4cb-b51b-4fd1-8501-aea1a032fb55 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.619528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926c4cb-b51b-4fd1-8501-aea1a032fb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.632553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 529/1072] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.685500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5926c4cb-b51b-4fd1-8501-aea1a032fb55 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.686693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5926c4cb-b51b-4fd1-8501-aea1a032fb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.779671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 544/1073] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:17.822542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.822853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.823130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.824481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 530/1074] 10.222.0.42 () {54 vars in 943 bytes} [Wed Oct 26 11:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:17.830574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:17.830910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:17.831191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:17.907363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] MFA Rules not processed for user `d256290285d44aafb8062ec2059b16ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:17.908835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.922449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:17.924006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:17.936614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:18.015261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 545/1075] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Oct 26 11:11:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:18.026105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:18.035437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8abf6-5197-4f8c-bc55-f246c793beb4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.073101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193f8e7-ffbe-408e-87fe-9b2094993d5d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:18.075907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:18.141917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.142792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.143118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.143386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.145290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.179020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:18.186100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:18.186703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:18.187009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:18.187279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:18.188847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:18.229504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.231114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfdad6-11bf-4d12-bb81-36d8b2b1f969 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.239795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 546/1076] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:11:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:11:18.284928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:18.286506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e681a5b-280d-44a6-b270-803fb78759a5 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:18.304321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 531/1077] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:20.619701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.620271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.620271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.621589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 547/1078] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:20.626809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d98b0-9a70-4cad-ad8b-27636abcb111 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:20.627257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d98b0-9a70-4cad-ad8b-27636abcb111 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:20.628728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d98b0-9a70-4cad-ad8b-27636abcb111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:20.639719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:20.712772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d41d98b0-9a70-4cad-ad8b-27636abcb111 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:20.714093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d41d98b0-9a70-4cad-ad8b-27636abcb111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:20.825240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 532/1079] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:21.119759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:21.119759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.285953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18565bf2-20f3-4f53-8d6c-a43de9bd9332 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:21.319787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:21.421265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:21.421884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:21.422162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:21.422440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:21.424073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:21.523698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:21.525848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bfc9d-7178-446b-922f-e3305236e5be service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:21.546349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 548/1080] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:23.943763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.944695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.945450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:23.947428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 533/1081] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:23.952826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929df5d-4911-404e-a8df-d86b71ad65ba None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:23.953255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929df5d-4911-404e-a8df-d86b71ad65ba None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:23.953357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929df5d-4911-404e-a8df-d86b71ad65ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.000099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:24.027944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0929df5d-4911-404e-a8df-d86b71ad65ba None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:24.029153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0929df5d-4911-404e-a8df-d86b71ad65ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.136777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 549/1082] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:24.434311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:24.435870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.533614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:24.606771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25c538-1ce7-43ec-8155-dcdf8fedfdf8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:24.738937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:24.739851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:24.740291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:24.740903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:24.743236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:24.799748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:24.845138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:24.847053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3ea4a-8ebc-44da-8d67-39e6c6a4185a service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:24.870879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 534/1083] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:27.176292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.179050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.179050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.179050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 550/1084] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:27.183554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894ab5-51a8-4813-b571-b1d4127849a5 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.183991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894ab5-51a8-4813-b571-b1d4127849a5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.184160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82894ab5-51a8-4813-b571-b1d4127849a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.272832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-82894ab5-51a8-4813-b571-b1d4127849a5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:27.274596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-82894ab5-51a8-4813-b571-b1d4127849a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.406424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 535/1085] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:27.692293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:27.693795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:27.813688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:27.858726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96cf8-0cfd-40ce-8514-362cbea6882f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:27.983548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:27.984173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:27.984564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:27.984913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:27.986605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:28.065334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:28.084635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:28.086393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc15bd-d8e9-41eb-8fee-ed13a12f0c7f service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:28.110074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 551/1086] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:37.404890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:38.106805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=13 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:41.873377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089eff7-87c5-43de-b2cb-321a26a67bff None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.873959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089eff7-87c5-43de-b2cb-321a26a67bff None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.873959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089eff7-87c5-43de-b2cb-321a26a67bff None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.875296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 536/1087] 10.222.0.42 () {54 vars in 918 bytes} [Wed Oct 26 11:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:41.880526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b666ac-cb42-4c3b-9aa8-7a9c45e74919 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:41.881045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b666ac-cb42-4c3b-9aa8-7a9c45e74919 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:41.881151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b666ac-cb42-4c3b-9aa8-7a9c45e74919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:41.956924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d8b666ac-cb42-4c3b-9aa8-7a9c45e74919 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:41.958160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d8b666ac-cb42-4c3b-9aa8-7a9c45e74919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:42.061466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 552/1088] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Oct 26 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:42.075712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318afbd-16ec-430c-b194-086a28d10787 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:42.075992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318afbd-16ec-430c-b194-086a28d10787 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:42.076263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318afbd-16ec-430c-b194-086a28d10787 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:42.077576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 537/1089] 10.222.0.42 () {54 vars in 918 bytes} [Wed Oct 26 11:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:11:42.082089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131562-38e9-497d-9bd0-13dd00a7c580 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:42.082365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131562-38e9-497d-9bd0-13dd00a7c580 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:42.082631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131562-38e9-497d-9bd0-13dd00a7c580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:42.155089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:42.157626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c5131562-38e9-497d-9bd0-13dd00a7c580 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:42.158844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c5131562-38e9-497d-9bd0-13dd00a7c580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:42.270147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 553/1090] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Oct 26 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:11:44.844142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:44.844671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:44.844671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:44.846004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 538/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:44.851883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5aea-a247-4654-8ff8-69046b0b516a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:44.852291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5aea-a247-4654-8ff8-69046b0b516a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:44.852437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc5aea-a247-4654-8ff8-69046b0b516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:44.917924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6bcc5aea-a247-4654-8ff8-69046b0b516a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:44.919180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6bcc5aea-a247-4654-8ff8-69046b0b516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:44.972048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:45.013878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 554/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:45.305247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:45.306752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.452441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4068e-2b1e-4876-bb0c-a62332580df3 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:45.524072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:45.563038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:45.563574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:45.563852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:45.564127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:45.565699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:45.653795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:45.655413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbdf87-75af-46d2-b0db-1ec31db9404d service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:45.672834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 539/1093] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:46.799929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.800397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.800501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.801839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 555/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:46.807412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46959-b66e-42c0-99ac-6f33ad77f787 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:46.807825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46959-b66e-42c0-99ac-6f33ad77f787 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:46.807960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c46959-b66e-42c0-99ac-6f33ad77f787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:46.883165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d6c46959-b66e-42c0-99ac-6f33ad77f787 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:46.884375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d6c46959-b66e-42c0-99ac-6f33ad77f787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:46.910524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:46.981325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 540/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:47.140451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:47.141980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:47.286784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a495a-bf9e-4406-b092-b4bd7710ad0c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:47.327716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:47.401024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:47.401608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:47.401887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:47.402162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:47.403853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:47.492538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:47.494127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12dbfa-70cd-44c6-a918-a24e253c204b service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:47.512420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 556/1096] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:48.616456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:48.616986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:48.616986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:48.618381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 541/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:48.623523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2ee2-982b-4a2a-abda-5f5e4650264d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:48.623928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2ee2-982b-4a2a-abda-5f5e4650264d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:48.624079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af2ee2-982b-4a2a-abda-5f5e4650264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:48.677511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:48.698420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-38af2ee2-982b-4a2a-abda-5f5e4650264d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:48.699675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-38af2ee2-982b-4a2a-abda-5f5e4650264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:48.796779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 557/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:48.948748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:48.950207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:49.093815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17199682-643a-4040-8ba4-3fce058952c8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:49.105427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:49.203701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:49.204236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:49.204510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:49.204779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:49.206485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:49.294518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:49.296157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abab72-3671-4265-ab06-aa568998d623 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:49.314273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 542/1099] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:50.405827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.406245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.406362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.407784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 558/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:50.412607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412f22a-baed-41e1-b9b2-5a3caa7debdb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.413013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412f22a-baed-41e1-b9b2-5a3caa7debdb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.413137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412f22a-baed-41e1-b9b2-5a3caa7debdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.432599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:50.487853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a412f22a-baed-41e1-b9b2-5a3caa7debdb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:50.489056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a412f22a-baed-41e1-b9b2-5a3caa7debdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.583770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 543/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:50.732554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:50.733980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:50.843134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:50.879146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc300e-9250-4dc5-8536-d20d622ab7ee None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:50.991278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:50.991816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:50.992094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:50.992366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:50.993937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:51.067399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:51.080234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:51.081810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69fb28-4736-44ea-ae3f-51450353719f service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:51.100908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 559/1102] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:52.187881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:52.188482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:52.188482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:52.189886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 544/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:52.195283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc618c7-37f9-41d3-ae64-894b1a6ae63e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:52.195683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc618c7-37f9-41d3-ae64-894b1a6ae63e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:52.195815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc618c7-37f9-41d3-ae64-894b1a6ae63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:52.271882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9fc618c7-37f9-41d3-ae64-894b1a6ae63e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:52.273073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9fc618c7-37f9-41d3-ae64-894b1a6ae63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:52.366297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 560/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:52.518679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:52.520588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:52.605313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:52.667840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b203e-fd80-422e-b18f-4991d87378df None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:52.783252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:52.783815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:52.784098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:52.784368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:52.785965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:52.835635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:52.877198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:52.878781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69725361-f63d-4fff-a1af-888df2b569fe service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:52.896579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 545/1105] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:54.001669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:54.002178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:54.002178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:54.003515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 561/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:54.009021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da7ef-ddcd-4bc9-be6d-b6cdb6acf17f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:54.009419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da7ef-ddcd-4bc9-be6d-b6cdb6acf17f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:54.009573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3da7ef-ddcd-4bc9-be6d-b6cdb6acf17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:54.083423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-7f3da7ef-ddcd-4bc9-be6d-b6cdb6acf17f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:54.084654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7f3da7ef-ddcd-4bc9-be6d-b6cdb6acf17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:54.182393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 546/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:54.334077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:54.335541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:54.386047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:54.483517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cc82d-c155-4c7c-91b0-6e1c0f21cb7a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:54.594616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:54.595167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:54.595447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:54.595717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:54.597289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:54.604643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:54.684779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:54.686335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8485ca-bc60-4998-b648-cea9d9416269 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:54.703900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 562/1108] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:55.825528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.826132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.826132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.827428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 547/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:55.832325 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9772b9-662d-4a93-94be-bdd321411d17 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:55.832782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9772b9-662d-4a93-94be-bdd321411d17 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:55.832897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9772b9-662d-4a93-94be-bdd321411d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:55.908447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bb9772b9-662d-4a93-94be-bdd321411d17 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:55.909652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bb9772b9-662d-4a93-94be-bdd321411d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:55.975920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:56.004329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 563/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:56.152658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:56.154164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:56.171274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:56.296773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b75c1-f162-42ed-88cc-ebe053c64ac6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:56.382724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:56.407342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:56.407950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:56.408231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:56.408485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:56.410055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:56.498110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:56.499793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6e6f-60d8-4c17-9190-c7ce7fa7c6c2 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:56.517187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 548/1111] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:57.611791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.612365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.612365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.613658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 564/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:11:57.618491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d6d-9626-4649-8332-a3cf9266773f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:57.618950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d6d-9626-4649-8332-a3cf9266773f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:57.619069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d6d-9626-4649-8332-a3cf9266773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:57.693873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-fcc62d6d-9626-4649-8332-a3cf9266773f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:57.695107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fcc62d6d-9626-4649-8332-a3cf9266773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:57.737353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:57.792477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 549/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:57.943556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:57.944974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.087324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e96de-4681-4802-9863-2b933dfd9cf9 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:58.140832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:58.198756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:58.199309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:58.199727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:58.200008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:58.201597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:58.290012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:11:58.291624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d9dd3-4097-444f-9081-65434568a381 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:11:58.309035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 565/1114] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:11:59.406259 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.406875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.406875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.408180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 550/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:11:59.413538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2707330-a60d-46d7-bc78-52ccd272932b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.413947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2707330-a60d-46d7-bc78-52ccd272932b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.414087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2707330-a60d-46d7-bc78-52ccd272932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.489099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f2707330-a60d-46d7-bc78-52ccd272932b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:11:59.490902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f2707330-a60d-46d7-bc78-52ccd272932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.492874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:59.585872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 566/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:11:59.735613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:11:59.737113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:11:59.884097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1abc89-703b-4f7f-b157-7fe336d233f2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:11:59.910527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:11:59.994335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:11:59.994886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:11:59.995166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:11:59.995435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:11:59.997019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:00.082922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:00.085017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044e41-94ff-441d-b8b6-e5b92e11e854 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:00.102896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 551/1117] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:02.149709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.150257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.150257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.151622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 567/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:02.157156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67c19-37cb-4d89-8614-b7b538ce6c1e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.157589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67c19-37cb-4d89-8614-b7b538ce6c1e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.157693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f67c19-37cb-4d89-8614-b7b538ce6c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.198383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:02.223538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-59f67c19-37cb-4d89-8614-b7b538ce6c1e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:02.225388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-59f67c19-37cb-4d89-8614-b7b538ce6c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.322424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 552/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:02.607482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:02.609265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.711255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f0dad8-2289-4976-8490-9171e021cf52 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:02.743881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:02.784247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:02.784851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:02.785126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:02.785395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:02.787091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:02.894950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:02.896378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd816c74-21cb-41c6-90ce-55a0bce5c701 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:02.913816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 568/1120] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:05.243357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.243928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.243928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.245283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 553/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:12:05.250634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff11c-277d-4db6-9375-3e57ec00b8f7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.251059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff11c-277d-4db6-9375-3e57ec00b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.251221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cff11c-277d-4db6-9375-3e57ec00b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.316722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-02cff11c-277d-4db6-9375-3e57ec00b8f7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:05.317932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-02cff11c-277d-4db6-9375-3e57ec00b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.330390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:05.413347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 569/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:05.699278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:05.700865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.803999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0823b5a-5650-4df2-a1f0-e39f8520fef9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:05.816543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:05.877577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:05.878175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:05.878453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:05.878721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:05.880329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:05.986874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:05.988238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036b1f9-c1f2-48e3-ba06-a8cb458c086c service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:06.005765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 554/1123] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:08.498415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.498976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.499095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.500365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 570/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:08.506141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f7b-60f1-49e8-8c17-6b955f45959a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:08.506545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f7b-60f1-49e8-8c17-6b955f45959a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:08.506701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e22f7b-60f1-49e8-8c17-6b955f45959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:08.553644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:08.572081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c7e22f7b-60f1-49e8-8c17-6b955f45959a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:08.573279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c7e22f7b-60f1-49e8-8c17-6b955f45959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:08.668995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 555/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:08.953882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:08.955514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:09.055701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2de76-46da-404c-aa29-5814c7e0b177 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:09.094468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:09.126895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:09.127490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:09.127768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:09.128033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:09.130174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:09.235892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:09.237245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85aa6-4e2c-4aee-a9af-a40b5b894ec9 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:09.255681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 571/1126] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:11.527271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.527874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.527874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.529194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 556/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:11.534347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae5cf-74e5-49b8-bb52-d6f1fe861466 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:11.534750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae5cf-74e5-49b8-bb52-d6f1fe861466 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:11.534901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ae5cf-74e5-49b8-bb52-d6f1fe861466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:11.600088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b81ae5cf-74e5-49b8-bb52-d6f1fe861466 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:11.601282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b81ae5cf-74e5-49b8-bb52-d6f1fe861466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:11.621245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:11.695342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 572/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:11.982135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:11.984034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:12.086436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e4a6-6fcc-48a4-9817-651a95045cd8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:12.105811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:12.159294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:12.159891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:12.160663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:12.160964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:12.162539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:12.270432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:12.271819 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3695805-a800-4741-b18f-a8a9435da5a3 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:12.290431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 557/1129] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:14.492609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.493126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.493235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.494503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 573/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:14.499898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b4bf-c45f-4b82-9095-fd10b7c07f14 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:14.500333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b4bf-c45f-4b82-9095-fd10b7c07f14 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:14.500430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267b4bf-c45f-4b82-9095-fd10b7c07f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:14.562569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:14.565328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f267b4bf-c45f-4b82-9095-fd10b7c07f14 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:14.566587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f267b4bf-c45f-4b82-9095-fd10b7c07f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:14.659836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 558/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:14.945606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:14.947185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:15.048537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b9b6b-0c93-4307-9f20-494afae75f47 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:15.096782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:15.121280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:15.121877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:15.122157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:15.122430 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:15.124135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:15.231667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:15.233343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb14058-380b-4bc3-be77-f82cb97e9740 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:15.251347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 574/1132] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:17.461961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:17.462523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:17.462523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:17.463887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 559/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:17.469075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759663-8e11-4951-992b-da6948b216ab None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:17.469458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759663-8e11-4951-992b-da6948b216ab None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:17.469651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71759663-8e11-4951-992b-da6948b216ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:17.534794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-71759663-8e11-4951-992b-da6948b216ab None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:17.535951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-71759663-8e11-4951-992b-da6948b216ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:17.564975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:17.631512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 575/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:17.916906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:17.918488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.020816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260de08d-159d-4c15-8d50-0a3b70dbb64e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:18.047778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:18.092473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:18.093067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:18.093342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:18.093612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:18.095175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:18.201073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:18.202425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a319e266-b2a9-4972-9951-e034e380ea2a service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:18.220327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 560/1135] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:20.416566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a744723-33cb-488b-ade7-424930935661 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:20.417129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a744723-33cb-488b-ade7-424930935661 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:20.417129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a744723-33cb-488b-ade7-424930935661 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:20.418462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 576/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:20.424029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b56c0-15f6-4f40-898c-3963cf4b6d58 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:20.424398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b56c0-15f6-4f40-898c-3963cf4b6d58 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:20.424398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b56c0-15f6-4f40-898c-3963cf4b6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:20.489644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a11b56c0-15f6-4f40-898c-3963cf4b6d58 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:20.490849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a11b56c0-15f6-4f40-898c-3963cf4b6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:20.499048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:20.587504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 561/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:12:20.872790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a744723-33cb-488b-ade7-424930935661 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:20.874379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1a744723-33cb-488b-ade7-424930935661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:20.976892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a744723-33cb-488b-ade7-424930935661 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:21.031251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:21.048732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:21.049342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:21.049624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:21.049900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:21.051578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:21.158713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:21.160223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8f40-3e22-4b2f-af4e-542dbad72337 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:21.177905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 577/1138] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:23.376378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.376943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.376943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.378792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 562/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:23.384154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a58ec-1b0e-4986-b31b-bbd8c2ba3f9f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:23.384599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a58ec-1b0e-4986-b31b-bbd8c2ba3f9f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:23.384729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a58ec-1b0e-4986-b31b-bbd8c2ba3f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:23.449670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b66a58ec-1b0e-4986-b31b-bbd8c2ba3f9f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:23.450887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b66a58ec-1b0e-4986-b31b-bbd8c2ba3f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:23.488580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:23.545610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 578/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:23.826180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:23.827734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:23.928729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3a85f-1ba5-46e3-abaa-7e5780c65649 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:23.966469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:24.001869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:24.002394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:24.002667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:24.003486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:24.005080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:24.109029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:24.110446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f283dd6-a14f-43d2-8d3d-9dc0229545df service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:24.129405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 563/1141] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:26.301345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.301943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.301943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.303273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 579/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:26.308604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd70ee-c05e-44aa-ae27-1b177d3bdee9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.309057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd70ee-c05e-44aa-ae27-1b177d3bdee9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.309164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd70ee-c05e-44aa-ae27-1b177d3bdee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.375000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e4dd70ee-c05e-44aa-ae27-1b177d3bdee9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:26.376282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e4dd70ee-c05e-44aa-ae27-1b177d3bdee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.395216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:26.471821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 564/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:26.754928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:26.756463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:26.858421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a82db-905d-4dba-a451-b6d08dc9ac49 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:26.923324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:26.929611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:26.930207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:26.930483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:26.930750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:26.932334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:27.037292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:27.038623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d9ac-6d2d-422d-8447-686d014d0701 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:27.056117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 580/1144] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:29.209521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.210097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.210097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.211500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 565/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:29.217053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75e5bc-6933-4881-af12-b0e88bc59495 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.217432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75e5bc-6933-4881-af12-b0e88bc59495 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.217605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb75e5bc-6933-4881-af12-b0e88bc59495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.283017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-cb75e5bc-6933-4881-af12-b0e88bc59495 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:29.284910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cb75e5bc-6933-4881-af12-b0e88bc59495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:29.332295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:29.380454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 581/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:12:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:29.661458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:29.663066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:29.766853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8e066-f8d6-46b3-b355-c10d461e3ce2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:29.816319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:29.841894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:29.842933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:29.843405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:29.843895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:29.845853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:29.951129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:29.952540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1efc03-d145-49d4-8dbb-894c4b3d3924 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:29.970955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 566/1147] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:39.378947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=22 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:39.968737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=66 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:49.798518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509b6b-6ee0-457a-beaf-570a8363fd4a None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:49.799105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509b6b-6ee0-457a-beaf-570a8363fd4a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:49.799105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509b6b-6ee0-457a-beaf-570a8363fd4a None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:49.800453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 582/1148] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:49.805766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa76d-f370-4457-aa0e-1e8f5e6b2a3a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:49.806166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa76d-f370-4457-aa0e-1e8f5e6b2a3a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:49.806311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aa76d-f370-4457-aa0e-1e8f5e6b2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:49.871553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-048aa76d-f370-4457-aa0e-1e8f5e6b2a3a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:49.872765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-048aa76d-f370-4457-aa0e-1e8f5e6b2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:49.967301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 567/1149] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:50.263363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.263659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.263934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.265267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 583/1150] 10.222.0.42 () {54 vars in 962 bytes} [Wed Oct 26 11:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:50.270427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98546-8a7e-4a23-9b18-4ba4688aaf18 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.270710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98546-8a7e-4a23-9b18-4ba4688aaf18 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.271012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98546-8a7e-4a23-9b18-4ba4688aaf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.344478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d9b98546-8a7e-4a23-9b18-4ba4688aaf18 None None] MFA Rules not processed for user `59ac752182c14071999766921f6b4f09`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:50.346184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d9b98546-8a7e-4a23-9b18-4ba4688aaf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.366922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:50.421333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 568/1151] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Oct 26 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:50.430074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:50.431649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.533358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62568aed-f7e1-4625-9609-434008d9abe7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:50.603740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:50.604338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:50.604614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:50.604881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:50.606716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:50.658771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:50.712288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:50.713706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdba25e-6aa4-4446-9a5f-e9c37c81121c service glance] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:50.733243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 584/1152] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:52.987237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.987758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.987758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:52.989162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 569/1153] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:52.994343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d6e0d-5c68-4783-8c0d-92e7310d4ae6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:52.994771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d6e0d-5c68-4783-8c0d-92e7310d4ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:52.994937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d6e0d-5c68-4783-8c0d-92e7310d4ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:53.059725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-869d6e0d-5c68-4783-8c0d-92e7310d4ae6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:53.060841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-869d6e0d-5c68-4783-8c0d-92e7310d4ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:53.155175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 585/1154] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:53.428564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:53.430101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:53.553385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203b155-b9ad-496d-bfd6-78ea1acdc2c7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:53.601553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:53.644637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:53.645230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:53.645503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:53.645771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:53.646389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:53.647510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:12:53.647510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:12:53.647510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:12:53.648689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471eccc-bbcb-47f1-a389-f27ffbd65ed9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:53.667727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 570/1155] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Oct 26 11:12:53 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:12:55.827227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:55.827760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:55.827760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:55.829116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 586/1156] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:55.834570 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eea242-6922-440d-bb51-571b117fa2b5 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:55.835038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eea242-6922-440d-bb51-571b117fa2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:55.835160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eea242-6922-440d-bb51-571b117fa2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:55.900347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-59eea242-6922-440d-bb51-571b117fa2b5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:55.901555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-59eea242-6922-440d-bb51-571b117fa2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:55.996108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 571/1157] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:56.296780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:56.298293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:56.308029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:56.442143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73eb6b-24ff-45c9-af11-4ed6ce78ff51 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:56.520694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:56.552001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:56.552552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:56.552829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:56.553096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:56.554656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:56.641097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:56.643311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bccf89-f9c6-494b-a225-87adc6bd7bc4 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:56.660773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 587/1158] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:12:59.016770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.017296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.017296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.018571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 572/1159] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:12:59.024109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1d4f-f29b-45d9-bbe1-64da1af09d21 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.024558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1d4f-f29b-45d9-bbe1-64da1af09d21 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.024656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da1d4f-f29b-45d9-bbe1-64da1af09d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.089055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c9da1d4f-f29b-45d9-bbe1-64da1af09d21 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:12:59.090251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c9da1d4f-f29b-45d9-bbe1-64da1af09d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.135339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:59.185511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 588/1160] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:12:59.475681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:12:59.477272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.501494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:59.577873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1f4af-b0c7-429b-8e4e-b8cff2258918 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:12:59.651436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:12:59.652024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:12:59.652297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:12:59.652565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:12:59.654126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:12:59.725876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:12:59.763296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:12:59.764734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f641d83-8b3b-4737-bab3-a05b47a37aa6 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:12:59.782348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 573/1161] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:12:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:01.969097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.969667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.969667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:01.970932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 589/1162] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:13:01.976223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9845d-c855-4792-8781-5eaf7fe1d764 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:01.976621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9845d-c855-4792-8781-5eaf7fe1d764 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:01.976859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9845d-c855-4792-8781-5eaf7fe1d764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:02.042424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-35d9845d-c855-4792-8781-5eaf7fe1d764 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:02.043652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-35d9845d-c855-4792-8781-5eaf7fe1d764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:02.137427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 574/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:02.425530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:02.427126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:02.530591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22ac16-9f15-4e6e-a3cd-b1e892e7f7e6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:02.600674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:02.602110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:02.602708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:02.603025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:02.603299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:02.605128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:02.711605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:02.713010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f9a0-ce7f-47e4-b719-21a956883df2 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:02.731617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 590/1164] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:04.962708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.963290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.963290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:04.964643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 575/1165] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:13:04.970002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d789eb-23bb-4e6d-b210-338ce1a6235d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:04.970442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d789eb-23bb-4e6d-b210-338ce1a6235d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:04.970550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d789eb-23bb-4e6d-b210-338ce1a6235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:05.035543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-17d789eb-23bb-4e6d-b210-338ce1a6235d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:05.036679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-17d789eb-23bb-4e6d-b210-338ce1a6235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.088131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:05.131639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 591/1166] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:05.420868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:05.423033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.458440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:05.523758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2cce-b361-4376-b248-fe6590c5e893 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:05.596037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:05.596627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:05.596908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:05.597173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:05.598721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:05.675673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:05.704482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:05.706287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1282bb-1fd0-467e-880d-35503577141f service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:05.725326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 576/1167] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:08.213646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.214211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.214211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.215654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 592/1168] 10.222.0.42 () {54 vars in 927 bytes} [Wed Oct 26 11:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:13:08.220858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3cfb2-d2d9-46e0-8c10-84eec800d991 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.221260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3cfb2-d2d9-46e0-8c10-84eec800d991 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.221412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3cfb2-d2d9-46e0-8c10-84eec800d991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.287528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cec3cfb2-d2d9-46e0-8c10-84eec800d991 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:13:08.288734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cec3cfb2-d2d9-46e0-8c10-84eec800d991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.384011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 577/1169] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Oct 26 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:13:08.676173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:13:08.677706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.780095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae687c-634d-4413-b3bf-9d52e9e67d3d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:13:08.851085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:13:08.851678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:13:08.851954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:13:08.852218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:13:08.853786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:13:08.862311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:08.960992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:13:08.962343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa7f3-5864-4628-9685-aab0f72ad1dc service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:13:08.981176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 593/1170] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:13:18.382284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=82 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:13:18.978884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=50 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:30.354935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:30.355535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:30.355535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:30.430729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:30.431952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:30.527475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 578/1171] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:30.535237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:30.535694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:30.535797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:30.610689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:30.611906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:30.705334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 594/1172] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:30.709960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:30.711257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:30.722974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f373-3227-42cc-baa7-2bad3b4f748f None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:30.784497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:30.817557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:30.818268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:30.818606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:30.818902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:30.819573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:14:30.819870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:30.820904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:14:30.820904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:14:30.820904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:14:30.821825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:30.833178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 579/1173] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:14:30.843133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:30.853356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b08d1-8677-404a-afe8-56f95862d645 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:30.920268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:30.945954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:14:30.950524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:14:30.951153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:31.059973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 595/1174] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:14:30 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:14:31.064288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:31.065775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:31.098885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d1481-9f31-49e0-8761-a8842b8fdbd0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:31.189447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:31.189967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:31.190304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:31.190581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:31.191237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:31.192231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:14:31.192231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:14:31.192231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:14:31.193414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:31.288566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 580/1175] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:14:31 2022] POST /identity/v3/users => generated 290 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:14:31.292800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:31.294308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:31.355746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09074a70-6ba0-4bd0-84a3-61bec7039335 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:31.435634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:31.446027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:31.446507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:31.446781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:31.447077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:31.447707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:31.449280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:31.462441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 596/1176] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:14:31.466518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:31.468101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:31.479510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d708ab-060e-456b-9b95-17219297cf25 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:31.504181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:31.573540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:31.574139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:31.574413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:31.574678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:31.575342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:31.576932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cfb899-7c16-460b-aba3-23f9dbd8c965 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:31.589512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 581/1177] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:14:31.593678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:31.595264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:31.628197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394f2af-75c2-41bf-9170-5e6ce6234466 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:31.720363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:31.720938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:31.721215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:31.721483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] PATH_INFO: `/v3/projects/12109c5f787f40fe98a92bec31c92165/users/c8785e51cc224c84a65f60ba55f5409c/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:31.744833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12109c5f787f40fe98a92bec31c92165, user_id=c8785e51cc224c84a65f60ba55f5409c, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:31.747552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d2c28-ee3f-443a-a5ed-6e0c6cf313d0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:31.843417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 597/1178] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:14:31 2022] PUT /identity/v3/projects/12109c5f787f40fe98a92bec31c92165/users/c8785e51cc224c84a65f60ba55f5409c/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:31.854502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3b6e-80a6-4f23-9bab-42a4d50e4650 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:31.854788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3b6e-80a6-4f23-9bab-42a4d50e4650 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:31.855100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b3b6e-80a6-4f23-9bab-42a4d50e4650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:31.910988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-591b3b6e-80a6-4f23-9bab-42a4d50e4650 None None] MFA Rules not processed for user `c8785e51cc224c84a65f60ba55f5409c`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:31.912202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-591b3b6e-80a6-4f23-9bab-42a4d50e4650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:31.989871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 582/1179] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:14:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:32.003340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:32.003623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:32.003899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:32.005217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 598/1180] 10.222.0.42 () {54 vars in 963 bytes} [Wed Oct 26 11:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:14:32.009938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a013f3-faac-46bd-96c8-5e4084d9c572 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:32.010219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a013f3-faac-46bd-96c8-5e4084d9c572 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:32.010489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a013f3-faac-46bd-96c8-5e4084d9c572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:32.041153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:32.084253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d4a013f3-faac-46bd-96c8-5e4084d9c572 None None] MFA Rules not processed for user `e3b32b5787164f1ba63e5e1e53f78174`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:32.085440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d4a013f3-faac-46bd-96c8-5e4084d9c572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:32.157912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 583/1181] 10.222.0.42 () {58 vars in 1060 bytes} [Wed Oct 26 11:14:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:32.166155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:32.167749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:32.216155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:32.268578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fc052-86f0-41ac-a9b4-ed02c559bf0c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:32.340357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:32.340962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:32.341245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:32.341520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:32.343133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:32.440044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:32.441464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:32.444883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:32.458786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 599/1182] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:14:32.505450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5639-d0ee-4561-9f8c-f7e9c2491cfb None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:32.505811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5639-d0ee-4561-9f8c-f7e9c2491cfb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:32.505917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5639-d0ee-4561-9f8c-f7e9c2491cfb None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:32.507331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 584/1183] 10.222.0.42 () {52 vars in 830 bytes} [Wed Oct 26 11:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:14:32.510792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:32.512033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:32.608924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b8e53-1722-48c5-9cf8-148f9131b129 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:32.708092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:32.708714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:32.708994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:32.709268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] PATH_INFO: `/v3/users/c8785e51cc224c84a65f60ba55f5409c` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:32.723974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8785e51cc224c84a65f60ba55f5409c)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:32.724987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:14:32.724987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:14:32.724987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:14:32.726203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:32.736331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:32.919627 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8785e51cc224c84a65f60ba55f5409c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:32.933218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8785e51cc224c84a65f60ba55f5409c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:32.933772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8785e51cc224c84a65f60ba55f5409c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:33.036152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8785e51cc224c84a65f60ba55f5409c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:33.036663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8785e51cc224c84a65f60ba55f5409c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:33.065464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 600/1184] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:14:32 2022] DELETE /identity/v3/users/c8785e51cc224c84a65f60ba55f5409c => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:33.069933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:33.070703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:33.071017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:33.127078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:14:33.128271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:33.202893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:33.214623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 585/1185] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:14:33.224573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:33.225841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:33.333836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0e676-abfe-4b4e-ac22-94d3134c05d7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:33.386353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:33.404649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:33.405250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:33.405524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:33.405789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:33.407384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:33.514630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:33.516050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb1a5d-eeb5-4b8a-ad81-66d942412357 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:33.533709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 601/1186] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:14:33.992460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:14:33.993762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:14:34.005446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7dd0-b16c-4a2a-9e37-30cddd262521 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:14:34.099523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:14:34.100146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:14:34.100421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:14:34.100687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] PATH_INFO: `/v3/projects/12109c5f787f40fe98a92bec31c92165` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:14:34.110912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=12109c5f787f40fe98a92bec31c92165)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:14:34.111948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:14:34.111948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:14:34.111948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:14:34.113129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:14:34.228369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12109c5f787f40fe98a92bec31c92165', 'request_id': 'req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:34.241653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12109c5f787f40fe98a92bec31c92165', 'request_id': 'req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:34.247654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12109c5f787f40fe98a92bec31c92165', 'request_id': 'req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:34.333273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] The token cache is being invalidate because project 12109c5f787f40fe98a92bec31c92165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:14:34.333493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:34.334013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1be78c2-3f31-4028-b27f-2cdfd5b7e9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:14:34.335361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 586/1187] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:14:33 2022] DELETE /identity/v3/projects/12109c5f787f40fe98a92bec31c92165 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:14:43.531524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=62 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:14:44.331183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:16:01.812777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:01.813292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:01.813394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:01.814714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 602/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:01.819668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60c71-4bc1-4df6-9852-3894498ad02b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:01.820092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60c71-4bc1-4df6-9852-3894498ad02b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:01.820193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60c71-4bc1-4df6-9852-3894498ad02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:01.900506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-05e60c71-4bc1-4df6-9852-3894498ad02b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:01.901735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-05e60c71-4bc1-4df6-9852-3894498ad02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:02.003332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 587/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:02.157458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:02.163506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:02.277969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d18bd-ca1b-4149-a277-52e6c41158a2 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:02.283333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:02.283938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:02.284221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:02.284490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:02.292423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:02.293993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff75a66-6517-46ad-84a1-9ff4a8eeae14 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:02.303701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 603/1190] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:03.366900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ccc0-0b92-4409-b2e5-224d65a3744e None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:03.367444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ccc0-0b92-4409-b2e5-224d65a3744e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:03.367444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371ccc0-0b92-4409-b2e5-224d65a3744e None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:03.368921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 588/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:03.374110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fd1d1-ffd2-4efd-93d6-984bd4662e4d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:03.374528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fd1d1-ffd2-4efd-93d6-984bd4662e4d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:03.374661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fd1d1-ffd2-4efd-93d6-984bd4662e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:03.395763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bb2fd1d1-ffd2-4efd-93d6-984bd4662e4d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:03.397023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bb2fd1d1-ffd2-4efd-93d6-984bd4662e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:03.404833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 604/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:03.418501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33835-690f-4556-b7c8-72fcbd9b8438 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:03.418789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33835-690f-4556-b7c8-72fcbd9b8438 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:03.419092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33835-690f-4556-b7c8-72fcbd9b8438 None None] PATH_INFO: `/v3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:03.420359 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 589/1193] 10.222.0.55 () {54 vars in 925 bytes} [Wed Oct 26 11:16:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:16:03.426313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4b50-560b-4185-be86-3876cc68fa40 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:03.426602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4b50-560b-4185-be86-3876cc68fa40 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:03.426905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4b50-560b-4185-be86-3876cc68fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:03.486545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2abe4b50-560b-4185-be86-3876cc68fa40 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:03.487792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2abe4b50-560b-4185-be86-3876cc68fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:03.541127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:16:03.555516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc33835-690f-4556-b7c8-72fcbd9b8438 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:03.559590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 605/1194] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Oct 26 11:16:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:03.570812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc33835-690f-4556-b7c8-72fcbd9b8438 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:03.576220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:03.576816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:03.577096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:03.577368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:03.586045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:03.587699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdd45d-41f5-4636-8e1a-be96e11b4b88 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:03.597527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 590/1195] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:12.531196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045757-ba4b-4fc6-8135-68910572f1a0 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:12.531702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045757-ba4b-4fc6-8135-68910572f1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:12.531803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045757-ba4b-4fc6-8135-68910572f1a0 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:12.533089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 606/1196] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:12.538084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cbd82-2811-414a-9a21-3bc258b6a882 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:12.538499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cbd82-2811-414a-9a21-3bc258b6a882 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:12.538613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425cbd82-2811-414a-9a21-3bc258b6a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:12.557993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-425cbd82-2811-414a-9a21-3bc258b6a882 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:12.560133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-425cbd82-2811-414a-9a21-3bc258b6a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:12.568689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 591/1197] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:16:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:12.576187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25045757-ba4b-4fc6-8135-68910572f1a0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:12.589907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25045757-ba4b-4fc6-8135-68910572f1a0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:12.595500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:12.596086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:12.596365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:12.596638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:12.604672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:12.606234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f03ed-38fd-47d7-ac1e-673c1a2465c4 service placement] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:12.615652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 607/1198] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:15.060618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27b7ee-aa95-4425-9e93-a5c947068dad None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:15.061186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27b7ee-aa95-4425-9e93-a5c947068dad None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:15.061186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e27b7ee-aa95-4425-9e93-a5c947068dad None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:15.062530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 592/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:15.067864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a1c3-1758-4a44-b937-02e87551ff58 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:15.068294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a1c3-1758-4a44-b937-02e87551ff58 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:15.068399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8a1c3-1758-4a44-b937-02e87551ff58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:15.088120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4da8a1c3-1758-4a44-b937-02e87551ff58 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:15.089312 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4da8a1c3-1758-4a44-b937-02e87551ff58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:15.097043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 608/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:15.248845 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27b7ee-aa95-4425-9e93-a5c947068dad None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:15.263248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e27b7ee-aa95-4425-9e93-a5c947068dad None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:15.274733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:15.274733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:15.274733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:15.274733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:15.278552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:15.280181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4cfd8-47e3-4392-85e7-976728c899d3 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:15.289583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 593/1201] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:16.345692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbbf37-e00a-4b76-8b5d-f27dac18cc7d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:16.346886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbbf37-e00a-4b76-8b5d-f27dac18cc7d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:16.346886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbbf37-e00a-4b76-8b5d-f27dac18cc7d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:16.348142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 609/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 26 11:16:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:16.352795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2213-753a-457c-bea1-5ffd2c33fe89 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:16.353204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2213-753a-457c-bea1-5ffd2c33fe89 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:16.353347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2213-753a-457c-bea1-5ffd2c33fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:16.372805 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f9aa2213-753a-457c-bea1-5ffd2c33fe89 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:16.374007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f9aa2213-753a-457c-bea1-5ffd2c33fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:16.381737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 594/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 26 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:16.533103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbbf37-e00a-4b76-8b5d-f27dac18cc7d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:16.547169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbbf37-e00a-4b76-8b5d-f27dac18cc7d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:16.552567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:16.553162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:16.553440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:16.553710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:16.561742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:16.563165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919433a-9ee1-4489-9fad-fcbfe9526df4 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:16.572926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 610/1204] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:20.566407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:20.566847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:20.567068 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:20.568236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 595/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:20.573543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b71eb-8106-402e-84ae-3b6876bacece None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:20.573971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b71eb-8106-402e-84ae-3b6876bacece None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:20.574070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b71eb-8106-402e-84ae-3b6876bacece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:20.592959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2b2b71eb-8106-402e-84ae-3b6876bacece None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:20.594137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2b2b71eb-8106-402e-84ae-3b6876bacece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:20.602955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 611/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:20.891888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:20.893614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:20.927325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:16:20.972782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354012e3-523a-4f9f-987b-818ae2265f13 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:20.976906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:20.977510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:20.977789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:20.978062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:20.986959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:20.988392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998f413-c2b4-456f-b745-56d26a476640 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:20.997589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 596/1207] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:23.113884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:23.114585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:23.114585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:23.115814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 612/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:23.121631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179e3a-d259-48fe-b3a2-58a9886c1054 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:23.122051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179e3a-d259-48fe-b3a2-58a9886c1054 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:23.122171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179e3a-d259-48fe-b3a2-58a9886c1054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:23.142736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1b179e3a-d259-48fe-b3a2-58a9886c1054 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:23.143965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1b179e3a-d259-48fe-b3a2-58a9886c1054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:23.151808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 597/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:23.440508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:23.442238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:23.455276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0db8a8-90e1-4e37-9cae-5e5df30efe98 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:23.458762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad07e31-38fc-4c4d-a52b-cb3ada77649e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:23.494162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 613/1210] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:27.100603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:27.101142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:27.101237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:27.102576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 598/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:27.108063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df041a89-a5e1-4827-9922-327d07b77cdb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:27.108519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df041a89-a5e1-4827-9922-327d07b77cdb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:27.108624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df041a89-a5e1-4827-9922-327d07b77cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:27.127572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-df041a89-a5e1-4827-9922-327d07b77cdb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:27.129016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-df041a89-a5e1-4827-9922-327d07b77cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:27.137398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 614/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:27.445934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:27.447664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:27.527620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395bcb-3496-468b-80cc-346b59125c52 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:27.531340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:27.531956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:27.532236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:27.532511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:27.541584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:27.543077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f3e19-6e98-4af6-9ee1-fcec11bae16e service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:27.552395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 599/1213] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:29.704367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:29.704826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:29.704919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:29.706297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 615/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 26 11:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:16:29.711488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8e8d-d054-4cc9-9f07-df84ea190c04 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:29.711908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8e8d-d054-4cc9-9f07-df84ea190c04 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:29.712039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8e8d-d054-4cc9-9f07-df84ea190c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:29.732735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0eeb8e8d-d054-4cc9-9f07-df84ea190c04 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:16:29.733915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0eeb8e8d-d054-4cc9-9f07-df84ea190c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:29.741625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 600/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 26 11:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:16:30.038596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:16:30.040397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:16:30.053109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa099e-de82-4f89-870c-72d560aebc0c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:16:30.056586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:16:30.057169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:16:30.057444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:16:30.057712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:16:30.066273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:16:30.067703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f113ef-790c-440f-b130-20ae9aad3a7b service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:16:30.077200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 616/1216] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:16:39.727181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=63 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:16:40.072757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=37 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:53.824175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a88b-56fa-4bdf-98e9-50220583ff1b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:53.824827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a88b-56fa-4bdf-98e9-50220583ff1b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:53.824940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7a88b-56fa-4bdf-98e9-50220583ff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:53.851844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-04e7a88b-56fa-4bdf-98e9-50220583ff1b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:53.853062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-04e7a88b-56fa-4bdf-98e9-50220583ff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:53.872762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 601/1217] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:53.884179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:53.885135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:53.885588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:53.914000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:53.915272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:53.928086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 617/1218] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:53.933143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8026b-a39d-4c58-af37-ff9a49f3f183 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:53.933518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8026b-a39d-4c58-af37-ff9a49f3f183 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:53.933696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8026b-a39d-4c58-af37-ff9a49f3f183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:53.938349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:53.953097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9d8cc-e994-4491-bebb-f43398c1cdf3 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:53.958151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:53.958773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:53.959082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:53.959360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:53.960019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:53.960333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:53.961378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:17:53.961378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:17:53.961378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:17:53.962288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:53.967371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0ee8026b-a39d-4c58-af37-ff9a49f3f183 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:53.968643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0ee8026b-a39d-4c58-af37-ff9a49f3f183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:53.985639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 618/1219] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:17:53.988017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 602/1220] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:53.996135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:53.998798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:53.999108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:53.999386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.011901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc19c-d71f-4559-8f02-f422b23cabd8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.032336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.033151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.033440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.033713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.034673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.037412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.049657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.050877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.076219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 603/1221] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:17:54.081237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.094763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa019d28-f329-496a-b160-a96ab799a482 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.104309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.104936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.105216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.105875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.106539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.106857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.109059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.129182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 604/1222] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:17:54.141335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.153240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372551c7-1555-43ca-aaa2-a22610f0872f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.166304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.166956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.167332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.167611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.168301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.170624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:17:54.170624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:17:54.170624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:17:54.172112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455f2a9-afee-4747-90a2-77870442e002 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.189628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 619/1223] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:53 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:54.199180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.299295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 605/1224] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:17:54.300642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140111b-4846-4441-b1d6-d2789cbb2f91 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.300928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140111b-4846-4441-b1d6-d2789cbb2f91 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.301202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a140111b-4846-4441-b1d6-d2789cbb2f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.308532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f3b-029d-4885-bd8c-dce96733226b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.346853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a140111b-4846-4441-b1d6-d2789cbb2f91 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.346853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a140111b-4846-4441-b1d6-d2789cbb2f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.393874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.394923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.395409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.395895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.396883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.398127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:17:54.398127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:17:54.398127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:17:54.399952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb15a1-0ffc-42c7-83a6-0f88a095295d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.426148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 606/1225] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.427428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.427714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.427985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.448297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.451865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.458086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 607/1226] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.458849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.468074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b7687-ea67-4b2c-9de0-3c247266d533 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.472550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.476851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.476851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.476851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.476851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.476851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc78cc-aef6-47ae-a615-dc5d488b19ff admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.502250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 620/1227] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/users => generated 290 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:17:54.503379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5343d5-afb0-40c1-91d7-f92749e40aaf None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.503683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5343d5-afb0-40c1-91d7-f92749e40aaf None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.503960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5343d5-afb0-40c1-91d7-f92749e40aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.523096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7e5343d5-afb0-40c1-91d7-f92749e40aaf None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.526534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7e5343d5-afb0-40c1-91d7-f92749e40aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.532511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 621/1228] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.533704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7117e94-2331-472d-a191-21dc1af1f35f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.533987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7117e94-2331-472d-a191-21dc1af1f35f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.534257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7117e94-2331-472d-a191-21dc1af1f35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.553856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e7117e94-2331-472d-a191-21dc1af1f35f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.555012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e7117e94-2331-472d-a191-21dc1af1f35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.563462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 622/1229] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.564610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d345b-a326-4254-aae7-0cfcf4e79b84 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.564891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d345b-a326-4254-aae7-0cfcf4e79b84 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.565160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d345b-a326-4254-aae7-0cfcf4e79b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.577051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 608/1230] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:54.578938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8fde6-8127-445d-a404-1a4029bdab24 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.579671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8fde6-8127-445d-a404-1a4029bdab24 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.579671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8fde6-8127-445d-a404-1a4029bdab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.585913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ac0d345b-a326-4254-aae7-0cfcf4e79b84 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.587257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ac0d345b-a326-4254-aae7-0cfcf4e79b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.595598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 623/1231] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.597182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.597469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.597987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.600546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-97d8fde6-8127-445d-a404-1a4029bdab24 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.602064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-97d8fde6-8127-445d-a404-1a4029bdab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.611534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 609/1232] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.612810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f4fd-b73d-406a-af19-a1c2b808ce59 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.613592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f4fd-b73d-406a-af19-a1c2b808ce59 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.614037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5f4fd-b73d-406a-af19-a1c2b808ce59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.620374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.621540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.630245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 624/1233] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.636058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.636733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-55d5f4fd-b73d-406a-af19-a1c2b808ce59 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.637917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-55d5f4fd-b73d-406a-af19-a1c2b808ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.641099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7911f-73ce-4585-a51b-a4fc97b52558 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.645770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.646358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.646638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.646757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 610/1234] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.646941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.647645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.647916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.648243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.648515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.649365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2daf619-8d23-4942-9892-f6d8b1c3da7a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.661052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 625/1235] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:54.662555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c569-5a22-47e0-bc24-7277fb19952c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.662862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c569-5a22-47e0-bc24-7277fb19952c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.663142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8c569-5a22-47e0-bc24-7277fb19952c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.667936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.669174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.677661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 611/1236] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.679711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.683235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-afc8c569-5a22-47e0-bc24-7277fb19952c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.684676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-afc8c569-5a22-47e0-bc24-7277fb19952c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.689677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a69374-d26b-44a4-ab0c-34ccbe3c7e34 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.693858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 626/1237] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.694892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.695181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.695452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.695816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.696505 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.696946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.697431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.698076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.699759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b979120-d478-4b53-991a-a8ae402ec310 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.702928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:54.712836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 612/1238] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:54.712836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.712836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.712836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.714767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.716000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.726278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 627/1239] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.727116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.731968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:54.733698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:54.739881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5092759-6d99-4e49-b7fc-0da592307274 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.741731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 613/1240] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:54.742345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.746311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.747011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.747293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.747564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.748757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.749060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.750651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.756187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4453f-bdbd-48e9-923f-7c493e42afb3 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.761147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.761744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.762024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.762297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.762969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.763285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.772318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 628/1241] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.772318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.772572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.774591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426416c-4a4f-4bb8-b9fc-ced5cd1480c3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.779228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.780298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.780864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.781141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.781767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.783384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.784073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 614/1242] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.784970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.795240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0e3f-f676-46e4-9201-748db71c9fd5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.799985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.800155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 629/1243] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:54.800357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.800632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.800900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.801262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.801525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.801832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.803438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.811023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368584d-ec4d-4297-8fa8-4e2aec55b226 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.815337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 615/1244] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.815710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.816122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.816292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.816576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.816846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.817476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.817779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.819392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.826286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4523b-2b91-481d-8713-d0076ac3871d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.830760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.831370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.831644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.831908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.839237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 630/1245] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.839385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.839512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.839604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.844112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395c179-43a8-44b4-b54e-f8e3411fd3b7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.845371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 616/1246] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:54.846186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.849109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.849698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.849980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.850258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.850922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.851230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.852813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.855485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52c759-6588-4923-ac01-a956fcb5cd22 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.859927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.860490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.860765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.861036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.861649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:17:54.861950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.863524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.864907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 631/1247] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.865057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:54.867734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.876104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 617/1248] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:17:54.877094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:54.878439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0eca5-c47a-448b-8f96-009a2e3912f6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.883855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.884459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.885061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.885061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.885616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.886442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e119d-2b13-4ca2-9aa4-c1550e706845 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:54.887537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.891555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:54.892131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:54.892555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:54.892673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:54.893571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:54.896266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:54.971540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 632/1249] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:54.972364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.000719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 618/1250] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:54 2022] POST /identity/v3/projects => generated 393 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.001756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.034048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f5432-f456-4d1b-98c6-183d6b75ada6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.063030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc03e6-c74d-4e82-88bf-e3bc5eef6ec1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.068295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.068890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.069167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.069435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.070047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.071976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.090586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.091274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.091575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.091854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] PATH_INFO: `/v3/projects/2375e01650824224b673bea3d06af9eb/users/fbbd63f10220465c996e6c706f1ef4a9/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.105539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2375e01650824224b673bea3d06af9eb, user_id=fbbd63f10220465c996e6c706f1ef4a9, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.108328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.191768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 619/1251] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.191768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.198917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 633/1252] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:54 2022] PUT /identity/v3/projects/2375e01650824224b673bea3d06af9eb/users/fbbd63f10220465c996e6c706f1ef4a9/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:55.199707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.212896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:55.255573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f896b-6ddf-449e-a794-89f48986db4a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.260225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.261377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.261377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.261377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.261956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.262563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b10ce-916f-4793-b173-dfd74a548b63 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.264480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.267927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.268581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.268852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.269148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] PATH_INFO: `/v3/projects/076d5555468141fb8835eb9dc9317671/users/eb2f97e51a444736be06322347085cbb/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.284633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076d5555468141fb8835eb9dc9317671, user_id=eb2f97e51a444736be06322347085cbb, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.287370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.377562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 620/1253] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.377562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.383969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 634/1254] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:55 2022] PUT /identity/v3/projects/076d5555468141fb8835eb9dc9317671/users/eb2f97e51a444736be06322347085cbb/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:55.393738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.430163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:55.447243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a1efb-3a39-4b3c-a6f9-d86c0743a9f6 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.447761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6212e0d-96b0-4b42-ba0a-8f6dd6fc45a9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.451910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.452515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.452699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.452895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.453186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.453343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.453564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.453802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.453909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.454440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.455749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.456374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.568321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 635/1255] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.569125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.579851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 621/1256] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.580968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.629219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00913f-0ee9-468a-a28c-05fe32f8ee9d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.641798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7928d0a-2c0f-491e-bfc1-f7b2723c8f0a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.646671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.647607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.647888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.648451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.648821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.650734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.654219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:55.683522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.684121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.684395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.684662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.685300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.687948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89174c7-0972-4470-973f-1d74b1d29d59 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.733850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 622/1257] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.734667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.744307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683b07-006e-490c-b301-870d3516eac4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.749068 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.749685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.749961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.750231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.750867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.752766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd4f3a-43a8-4560-9a89-c7f42be8d3b8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.764050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 636/1258] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 294 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.768140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c837356-96fa-44da-968e-28eb18859fab None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.768140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c837356-96fa-44da-968e-28eb18859fab None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.768140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c837356-96fa-44da-968e-28eb18859fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.783944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2c837356-96fa-44da-968e-28eb18859fab None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:55.798980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2c837356-96fa-44da-968e-28eb18859fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:55.798980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 637/1259] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:55.798980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c837356-96fa-44da-968e-28eb18859fab None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.803684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c837356-96fa-44da-968e-28eb18859fab None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.808215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.808800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.809074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.809339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.809949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.812001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.853108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 623/1260] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.854369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.854651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.854963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.884924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] MFA Rules not processed for user `eb2f97e51a444736be06322347085cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:55.884924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:55.913714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 638/1261] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:55.913714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.923728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399976d8-a92a-4a7b-bda6-3558958ca572 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.928293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.928880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.929298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.929577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.930191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.932202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:55.951043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 624/1262] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:55.951854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:55.962521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1ee31-9de1-4952-bee1-85c81f376902 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:55.971234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:55.971234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:55.971234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:55.971234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:55.972351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:55.972351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.027593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 639/1263] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 281 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:56.027593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.037674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5472b-20ea-40fe-ab1d-db7c3d1d11fc admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.042261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.042872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.043159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.043450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.044098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.046299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.055507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 625/1264] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:55 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:56.055507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.060077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 640/1265] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.060077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.065352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e22dd-24bf-4297-9ada-7fc0d3ace0e2 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.069960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.070544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.070855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.071125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.071755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.073326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.079136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:56.084486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 626/1266] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.085401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.096424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f4832-3455-4b2c-8069-93d41a509a2e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.101291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.101958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.102238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.102510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.103179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.104745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.118091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 627/1267] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.118091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.119033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c72ff-617d-4e8d-9bee-4068ed14945d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.122622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.123333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.128301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.128301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.128495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf206f74-6a1d-44cc-945c-57079f1d261c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.129951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.130521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.130799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.131098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.131703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.133098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.133250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abcdcf-ab85-4e16-ad8a-9181b3a589d4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.134607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.143490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 628/1268] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.144632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 641/1269] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:56.144760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859f44-abf3-4bd7-876e-20803239b05c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.144843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859f44-abf3-4bd7-876e-20803239b05c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.145106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54859f44-abf3-4bd7-876e-20803239b05c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.145254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.146357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 629/1270] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:17:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:17:56.147086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859f44-abf3-4bd7-876e-20803239b05c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.160416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54859f44-abf3-4bd7-876e-20803239b05c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.161777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8210c-d03e-47d9-bc46-a393cf116112 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.165158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.165919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.166346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.166616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.166779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.167392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.167530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.167704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.168142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.168812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.169058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.170545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.180782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 630/1271] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.181380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 642/1272] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.181597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.182851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.193636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b558962-5833-4dc1-b1e0-e8ae98ab0c10 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.194895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab37c8-7092-47c7-b103-79aab1827af7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.198761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.199377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.199651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.199837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.199981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.200431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.200553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.200716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.201502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.202166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.202639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.203794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.217521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 643/1273] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.217521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.217956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 631/1274] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.218884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.224757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c15097d-a219-40a9-a28b-72c42c53e793 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.228143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d334bb-c263-4a10-b2f5-d6582260e47f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.231743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.231743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.231743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.231743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.232220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.233409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.233546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3579d9de-7134-4474-a41b-fbe32eab7333 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.233988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.234267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.234685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.235194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.236794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.244218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 644/1275] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.245287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.245576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.245855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.246725 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 632/1276] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.247562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.257773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946bbdb-4d9b-4264-8030-71ce1b5d48b3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.263593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.264382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.264663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.264935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.265809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.267436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.281450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 633/1277] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.282151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.291902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:56.295464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a91e5-f586-4699-812e-ad0c1f6be6f9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.295586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:56.298304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.298939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.299735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.300016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.301005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.302803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.312699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:56.319130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 634/1278] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:56.319130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.328470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5125-664e-4b37-8f42-b5c664e63994 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.333213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.333867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.334147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.334502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] PATH_INFO: `/v3/projects/9d9d2ce62fb74bdb8de53bc06b1a999d/users/ff0c5f8521534fc1a0a9c5ea28fbaac1/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.351645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d9d2ce62fb74bdb8de53bc06b1a999d, user_id=ff0c5f8521534fc1a0a9c5ea28fbaac1, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.356962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.365516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 645/1279] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:56.366324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.376564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca90c0-7d94-4ab3-809f-ff2fe59bad7b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.381037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.381653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.384194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.384194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] PATH_INFO: `/v3/projects/ab29bf15969d46da9670efa2b52e6676/users/41276c93e476436791b89ab8a0d4537d/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.396755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab29bf15969d46da9670efa2b52e6676, user_id=41276c93e476436791b89ab8a0d4537d, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.399405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.439851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 635/1280] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/9d9d2ce62fb74bdb8de53bc06b1a999d/users/ff0c5f8521534fc1a0a9c5ea28fbaac1/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.440650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.464947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 646/1281] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/ab29bf15969d46da9670efa2b52e6676/users/41276c93e476436791b89ab8a0d4537d/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.466243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.511711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0a69d-bec0-41e4-ab20-d5edbd655002 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.547489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53c61-7fdf-4f22-8ed6-c092ad42962f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.553600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.554277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.554552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.554868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] PATH_INFO: `/v3/projects/a1ec496558f34086b454e51eaadeca4e/users/81210ebe54df412da347c158904f9cb2/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.571314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ec496558f34086b454e51eaadeca4e, user_id=81210ebe54df412da347c158904f9cb2, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.574208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.578649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.579749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.579749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.579952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] PATH_INFO: `/v3/projects/724121c2144843b7adc7e17312613c9d/users/e031dd9da25146eebb8813dc8bad8b85/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.591223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:56.595640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=724121c2144843b7adc7e17312613c9d, user_id=e031dd9da25146eebb8813dc8bad8b85, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.598407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.696901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 647/1282] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/a1ec496558f34086b454e51eaadeca4e/users/81210ebe54df412da347c158904f9cb2/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.697885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.698841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 636/1283] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/724121c2144843b7adc7e17312613c9d/users/e031dd9da25146eebb8813dc8bad8b85/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.699816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.782305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80c083-b072-4471-96f7-c888f339852a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.800596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:56.802039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08da165-972a-4260-b9e5-944ce896cfbb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:56.808773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.810132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.810132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.810600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] PATH_INFO: `/v3/projects/0791e18214d94c2495a75a53ed80283f/users/cc6727ff272f47c8b30faf741e108e52/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.828513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0791e18214d94c2495a75a53ed80283f, user_id=cc6727ff272f47c8b30faf741e108e52, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.831203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.843926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:56.844592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.844860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.845165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] PATH_INFO: `/v3/projects/7c0fb82767b3409098903b6ba8dc7b62/users/da9a0e67d59b49d4aa945b4da77ad2af/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.860764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0fb82767b3409098903b6ba8dc7b62, user_id=da9a0e67d59b49d4aa945b4da77ad2af, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:56.863020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50453f67-6916-4e1f-8c1c-22e7481e8c0a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:56.933993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 637/1284] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/0791e18214d94c2495a75a53ed80283f/users/cc6727ff272f47c8b30faf741e108e52/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.934583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:56.963385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 648/1285] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:56 2022] PUT /identity/v3/projects/7c0fb82767b3409098903b6ba8dc7b62/users/da9a0e67d59b49d4aa945b4da77ad2af/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:56.965056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f156d3-6743-4e56-bc80-e660fd0f5617 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:56.965626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f156d3-6743-4e56-bc80-e660fd0f5617 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:56.965969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f156d3-6743-4e56-bc80-e660fd0f5617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:56.992554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-44f156d3-6743-4e56-bc80-e660fd0f5617 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:56.993770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-44f156d3-6743-4e56-bc80-e660fd0f5617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.009578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155f196-3287-4530-a253-f1c1d9c827d7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.057309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 649/1286] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.059091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55fd5d-110c-4e2d-95de-f383543e6fc6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.059386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55fd5d-110c-4e2d-95de-f383543e6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.059656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55fd5d-110c-4e2d-95de-f383543e6fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.078574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.079164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.079443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.079568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0a55fd5d-110c-4e2d-95de-f383543e6fc6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.079729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.080651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a55fd5d-110c-4e2d-95de-f383543e6fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.085677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:57.088477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 650/1287] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.089719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398fb0-3eb5-4c7a-af7c-14f54fa8fa12 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.090012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398fb0-3eb5-4c7a-af7c-14f54fa8fa12 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.090282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8398fb0-3eb5-4c7a-af7c-14f54fa8fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.118403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e8398fb0-3eb5-4c7a-af7c-14f54fa8fa12 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.119649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e8398fb0-3eb5-4c7a-af7c-14f54fa8fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.128629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 651/1288] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.138102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f387d2-81e8-4d96-90be-db78498fcc7e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.138480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f387d2-81e8-4d96-90be-db78498fcc7e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.139361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f387d2-81e8-4d96-90be-db78498fcc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.139639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.141435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf370d-0141-440e-ac60-403ee6cc84cc service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.152403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 638/1289] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:57.164495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.164768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.166611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.169220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-83f387d2-81e8-4d96-90be-db78498fcc7e None None] MFA Rules not processed for user `81210ebe54df412da347c158904f9cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.170495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-83f387d2-81e8-4d96-90be-db78498fcc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.203833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] MFA Rules not processed for user `cc6727ff272f47c8b30faf741e108e52`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.205482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.245204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 652/1290] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.246782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.247230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.247677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.277842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] MFA Rules not processed for user `da9a0e67d59b49d4aa945b4da77ad2af`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.279446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.285717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 639/1291] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.285717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.291026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:57.348451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242295f-cfea-4360-89ed-b95838790bbb None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.351446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 653/1292] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:57.352336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.352942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.353220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.353489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.355665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.363873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.365190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727b9b1-3a4c-460f-af5f-67b2806b3eb0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.365347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.370932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.371106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.371473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.372102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.377058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 640/1293] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:57.377973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.383327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.391445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.397447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 654/1294] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:57.398769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.406405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e3e17-4d6f-4e09-88f4-08dc89cc5b0e service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.409339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252006cf-614d-4ef0-af38-e185e5999db7 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.411639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.412267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.413109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.413404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.415822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.417252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.417702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.418144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.419089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.421542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.427839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.431944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.440977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 641/1295] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:57.441797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.452371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b46a2-df1b-490c-a9d2-2ec7f2da4012 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.460487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.460487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.460487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.460487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.460904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.460904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46723-f56f-4ede-ac18-cba8c8bbda74 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.615061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 655/1296] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/projects => generated 403 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:57.616321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.633033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 642/1297] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/projects => generated 365 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:57.634312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e384-cd61-49a9-a2e1-7bf46871027a None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.634591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e384-cd61-49a9-a2e1-7bf46871027a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.634917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070e384-cd61-49a9-a2e1-7bf46871027a None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.638778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 643/1298] 10.222.0.42 () {54 vars in 912 bytes} [Wed Oct 26 11:17:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:17:57.639562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070e384-cd61-49a9-a2e1-7bf46871027a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.652650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:57.691761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73c376-cfd6-426b-a482-aeadf0ec978f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.724956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070e384-cd61-49a9-a2e1-7bf46871027a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.728434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.729052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.729329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.730073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.731021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.735768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.781940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.782748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.783106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.783436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.784390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.786759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5546638-0024-4086-ad33-7413f03a47db admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.874095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 644/1299] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/users => generated 308 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:57.874095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:57.882217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dc94c-4ba5-4935-9221-1bd99a75abac admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:57.887816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:57.888592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.888962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.889303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.889996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:57.892435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:57.907979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 656/1300] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/projects => generated 381 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:57.907979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:57.907979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:57.907979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:57.924059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:57.930317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:57.931885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:57.994453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 645/1301] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:57.994453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.016577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e741-dfec-4475-ace3-8d6f4f52521d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.082970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 657/1302] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:17:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:58.086056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.182056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.183391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.184327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.184811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.185721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.188707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.212023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b6af9-b4c6-49d9-8201-9bbe575f5a44 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.219862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.220577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.220920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.221252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.221953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.228758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.241774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 646/1303] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:58.242663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.256350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7202c29-b4b4-45b6-9326-3cd0aee31cd7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.261254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.262122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.262765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.263117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.263942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.266202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.278993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 647/1304] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:58.280444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.322268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 658/1305] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:17:58 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:17:58.323257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.335570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09826248-1fc9-4ce5-8ffc-0e4d5d4061af admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.341669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.342737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.343422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.343756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] PATH_INFO: `/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f/users/f76979f4164649df923fe3ecaf621795/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.362899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc013e0eea54a2f90d5b5b10ecded3f, user_id=f76979f4164649df923fe3ecaf621795, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.365597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.369574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a99b9-472b-4c01-bd6e-65a22b57bd4c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.377733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.379296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.379696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.380077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.403056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.405737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.435524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 648/1306] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:58.436253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.446399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:58.455017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8026c-aa8e-4efc-90d7-9d9e08c773d1 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.460466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 659/1307] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:58 2022] PUT /identity/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f/users/f76979f4164649df923fe3ecaf621795/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:58.464640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.556466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.557580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.558064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.559054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] PATH_INFO: `/v3/projects/c4d92ee9add34bdda4c6df36abe7c787/users/4c491023f23142baa9e7531e4e409674/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.586390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d92ee9add34bdda4c6df36abe7c787, user_id=4c491023f23142baa9e7531e4e409674, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.590733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.616526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1eeff-0890-4eb7-899c-4ce2ede43542 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.627676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.629610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.632846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.633206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.641223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.647576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.653447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:58.661533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 660/1308] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:58.662807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.734071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 649/1309] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:58 2022] PUT /identity/v3/projects/c4d92ee9add34bdda4c6df36abe7c787/users/4c491023f23142baa9e7531e4e409674/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:58.734071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.736361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc945b-1b51-42a2-b1b7-666cf76b9c5e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.871231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598f43b-75ff-411e-b160-889c8228cf8c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.878560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.880204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.886618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.886618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] PATH_INFO: `/v3/projects/5e624ae9521e4b8191d8a86bff142951/users/289041fab15d428db7cd9b7af57fa7ea/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.899687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.901927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.902303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.903106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e624ae9521e4b8191d8a86bff142951, user_id=289041fab15d428db7cd9b7af57fa7ea, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.903482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.904148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.907743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.908751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:58.939286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 661/1310] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:58.944183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:58.965241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c0338-af72-46a1-8d74-a363c48fefdd admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:58.983846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:58.984566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:58.984853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:58.985133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:58.985974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:58.992238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.001269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 650/1311] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:58 2022] PUT /identity/v3/projects/5e624ae9521e4b8191d8a86bff142951/users/289041fab15d428db7cd9b7af57fa7ea/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:59.001269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.012492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 662/1312] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:59.013816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.026396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83413ea2-98a6-4b39-84d6-8ed8c6294060 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.110544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a7113-5d95-4067-b1be-a2600496f26e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.112110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.113335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.113845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.114578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.116149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.121587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.127960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.128989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.129687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.130073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] PATH_INFO: `/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f/users/f76979f4164649df923fe3ecaf621795/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.132595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc013e0eea54a2f90d5b5b10ecded3f, user_id=f76979f4164649df923fe3ecaf621795, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.134789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 663/1313] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:17:59.136038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.136653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.153323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 651/1314] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:59 2022] PUT /identity/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f/users/f76979f4164649df923fe3ecaf621795/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:59.155155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.155311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587a29-4406-4ef6-9ae1-25ccd7d68c2e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.167566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:59.244000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.245102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.245564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.245986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] PATH_INFO: `/v3/projects/c4d92ee9add34bdda4c6df36abe7c787/users/4c491023f23142baa9e7531e4e409674/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.248149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d92ee9add34bdda4c6df36abe7c787, user_id=4c491023f23142baa9e7531e4e409674, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.249577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3244f-d303-463d-8018-781d098154f4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.253731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384771f7-9a7f-474c-9974-b25667f3dff9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.254408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.255264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.255669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.256065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] PATH_INFO: `/v3/projects/5e624ae9521e4b8191d8a86bff142951/users/289041fab15d428db7cd9b7af57fa7ea/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.258231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e624ae9521e4b8191d8a86bff142951, user_id=289041fab15d428db7cd9b7af57fa7ea, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.261269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0acf3-a9a9-432e-a1b4-77dd6a3efc47 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.269602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 664/1315] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:59 2022] PUT /identity/v3/projects/c4d92ee9add34bdda4c6df36abe7c787/users/4c491023f23142baa9e7531e4e409674/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:59.272394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.273732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.274207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.284971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 652/1316] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:17:59 2022] PUT /identity/v3/projects/5e624ae9521e4b8191d8a86bff142951/users/289041fab15d428db7cd9b7af57fa7ea/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:17:59.294992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bfa3-b36f-495d-b309-3f1233f84a94 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.294992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bfa3-b36f-495d-b309-3f1233f84a94 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.294992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7bfa3-b36f-495d-b309-3f1233f84a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.331630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-94c7bfa3-b36f-495d-b309-3f1233f84a94 None None] MFA Rules not processed for user `4c491023f23142baa9e7531e4e409674`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:59.333572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-94c7bfa3-b36f-495d-b309-3f1233f84a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:59.334366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] MFA Rules not processed for user `f76979f4164649df923fe3ecaf621795`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:59.335605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:59.479860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 653/1317] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:59.481960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.482290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.482575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.500549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 665/1318] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:59.500549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.544786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:17:59.547849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] MFA Rules not processed for user `289041fab15d428db7cd9b7af57fa7ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:17:59.547849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:17:59.587742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c08d3-4f9c-4ebe-b5a7-34c832540944 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.591676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.592528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.592808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.593079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.604362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.606173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.618099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 666/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:59.640414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 654/1320] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:17:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:17:59.712283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.730756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd2f93-37e3-4d36-a4e9-f82dc236e572 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.738632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.739653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.740091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.740573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.743115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:17:59.758420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.758580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b279628-f1d0-40c4-a712-a2effd951afe None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:17:59.760782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.763006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:17:59.763831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:17:59.764801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:17:59.765786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:17:59.776484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:17:59.778224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:17:59.786178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 667/1321] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:17:59.789264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 655/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:17:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:00.057405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:00.219735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:00.238791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df1b76-f187-4597-9e78-73d523aa7759 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:00.254905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:00.254905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.254905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.254905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.284877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:00.297189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188252-9713-48d7-81d8-8687205040f8 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:00.306586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:00.308681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.314100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.314762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.414493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:00.417681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb52005-019d-44ce-ac8f-3cdb3a614ed5 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:00.430273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 668/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:00.432813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf60-33a2-4478-9727-bb5aea05eb6d None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.433294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf60-33a2-4478-9727-bb5aea05eb6d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.433746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf60-33a2-4478-9727-bb5aea05eb6d None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.444038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 669/1324] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:18:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:00.445723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5a13-6bd8-4a26-afd3-141e487a9f50 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.445999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5a13-6bd8-4a26-afd3-141e487a9f50 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.446339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f5a13-6bd8-4a26-afd3-141e487a9f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.463547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:00.468301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12867443-a78b-4cc5-ba34-64af8388b046 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:00.474478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-664f5a13-6bd8-4a26-afd3-141e487a9f50 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:00.475912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-664f5a13-6bd8-4a26-afd3-141e487a9f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:00.495538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 656/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:00.526159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:00.612731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 670/1326] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:18:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:00.620075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9956a3-e7b4-4bb5-9250-133abecfc777 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.620271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9956a3-e7b4-4bb5-9250-133abecfc777 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.620478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9956a3-e7b4-4bb5-9250-133abecfc777 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.623029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 657/1327] 10.222.0.42 () {54 vars in 912 bytes} [Wed Oct 26 11:18:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:00.636929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:00.637226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:00.637505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:00.703199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:00.708749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:00.726172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 671/1328] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:18:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:01.070460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9956a3-e7b4-4bb5-9250-133abecfc777 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:01.091821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9956a3-e7b4-4bb5-9250-133abecfc777 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:01.112876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:01.113509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:01.115648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:01.115935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:01.138478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:01.142610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53857d59-245e-4921-a0f6-5ca4be7eb3cd service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:01.171540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 658/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:01.296513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:01.386226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c417c0-3a64-42f4-b4a3-60aa1cd3c986 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:01.391583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:01.392343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:01.392729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:01.393106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] PATH_INFO: `/v3/users/81210ebe54df412da347c158904f9cb2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:01.394490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81210ebe54df412da347c158904f9cb2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:01.397730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:01.688457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81210ebe54df412da347c158904f9cb2'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:01.699661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81210ebe54df412da347c158904f9cb2'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:01.705294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81210ebe54df412da347c158904f9cb2'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:01.777425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81210ebe54df412da347c158904f9cb2'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:01.780082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81210ebe54df412da347c158904f9cb2'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:01.834035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 672/1330] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:01 2022] DELETE /identity/v3/users/81210ebe54df412da347c158904f9cb2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:01.839274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:01.839801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:01.840281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:01.889121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:01.891880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:02.036429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 659/1331] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:02.047820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:02.049304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:02.095623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:02.130582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14696f9d-44f7-42f5-8eea-336ab2b8ac3d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:02.135381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:02.136446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:02.137007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:02.137355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:02.157922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:02.160526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:02.176521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 673/1332] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:04.232502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:04.236575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:04.258189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d20164-44b5-4a87-a9cc-abb777edc981 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:04.273375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:04.274601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:04.274907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:04.275191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] PATH_INFO: `/v3/projects/a1ec496558f34086b454e51eaadeca4e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:04.280283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ec496558f34086b454e51eaadeca4e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:04.291653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:04.570705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ec496558f34086b454e51eaadeca4e', 'request_id': 'req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:04.588340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ec496558f34086b454e51eaadeca4e', 'request_id': 'req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:04.608286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ec496558f34086b454e51eaadeca4e', 'request_id': 'req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:04.734596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] The token cache is being invalidate because project a1ec496558f34086b454e51eaadeca4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:04.735680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:04.739903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:04.743607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 660/1333] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:04 2022] DELETE /identity/v3/projects/a1ec496558f34086b454e51eaadeca4e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:04.747351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:04.805180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3a41-db46-457f-9cde-fae0152a204e service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:05.085990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:05.087429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:05.087429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:05.090128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] PATH_INFO: `/v3/users/f76979f4164649df923fe3ecaf621795` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:05.096310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76979f4164649df923fe3ecaf621795)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:05.105539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:05.467717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76979f4164649df923fe3ecaf621795'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:05.489697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76979f4164649df923fe3ecaf621795'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:05.491340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76979f4164649df923fe3ecaf621795'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:05.580662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76979f4164649df923fe3ecaf621795'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:05.581414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5f3081ec-b644-424d-bad7-cb36b4a3213e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76979f4164649df923fe3ecaf621795'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:05.618293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 674/1334] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:04 2022] DELETE /identity/v3/users/f76979f4164649df923fe3ecaf621795 => generated 0 bytes in 871 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:06.430656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:06.430656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:06.451553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:06.465258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9e383-37d7-497b-8b9e-dae2f01d2bf4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:06.608539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:06.609597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:06.610063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:06.610526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] PATH_INFO: `/v3/users/eb2f97e51a444736be06322347085cbb` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:06.611943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2f97e51a444736be06322347085cbb)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:06.613212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: WARNING py.warnings [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:18:06.613212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: warnings.warn(msg) Oct 26 11:18:06.613212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: Oct 26 11:18:06.616324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:06.919470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2f97e51a444736be06322347085cbb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:06.943184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2f97e51a444736be06322347085cbb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:06.949329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2f97e51a444736be06322347085cbb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.029891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2f97e51a444736be06322347085cbb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.030907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2f97e51a444736be06322347085cbb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.108202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 661/1335] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:06 2022] DELETE /identity/v3/users/eb2f97e51a444736be06322347085cbb => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:07.113455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:07.113992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:07.114184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:07.155607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:07.157545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:07.264100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:07.292154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 675/1336] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:07.304828 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:07.306538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:07.423126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:07.424701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:07.441904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82b3b9b-fd57-434b-9782-40c3a9d71479 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:07.444626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61167df9-80fd-46d6-91a4-f0ae177ab801 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:07.459645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:07.460766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:07.461813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:07.462580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:07.463031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:07.463764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:07.464709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:07.465450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] PATH_INFO: `/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:07.468138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc013e0eea54a2f90d5b5b10ecded3f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:07.476735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: WARNING py.warnings [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 26 11:18:07.476735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: warnings.warn(msg) Oct 26 11:18:07.476735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: Oct 26 11:18:07.481645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:07.492935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:07.494673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a24e26-2505-457e-9025-4d5cfbed9fea service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:07.528740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 662/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:07.675523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:07.675943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:07.676221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:07.734030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc013e0eea54a2f90d5b5b10ecded3f', 'request_id': 'req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.773893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] MFA Rules not processed for user `41276c93e476436791b89ab8a0d4537d`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:07.777925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:07.782281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc013e0eea54a2f90d5b5b10ecded3f', 'request_id': 'req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.805734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc013e0eea54a2f90d5b5b10ecded3f', 'request_id': 'req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.968394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] The token cache is being invalidate because project 3fc013e0eea54a2f90d5b5b10ecded3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:07.968693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.971433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-baeb2dd7-db1f-4c22-afd0-2d4a4018aa9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:07.983362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 676/1338] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:07 2022] DELETE /identity/v3/projects/3fc013e0eea54a2f90d5b5b10ecded3f => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:07.984898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e222c7a-feb9-4219-a877-42d4c22cb843 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:07.985234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e222c7a-feb9-4219-a877-42d4c22cb843 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:07.985682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e222c7a-feb9-4219-a877-42d4c22cb843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:07.989996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 663/1339] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:08.013809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.017906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:08.081159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8e222c7a-feb9-4219-a877-42d4c22cb843 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:08.087924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8e222c7a-feb9-4219-a877-42d4c22cb843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:08.197239 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:08.251821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b98e22-4924-4dc2-977e-78981fc2df2f None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.258988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:08.259977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.260414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.260890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.265667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:08.280601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:08.298929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:08.321879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 677/1340] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:08.337758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 664/1341] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:08.352147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.355217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.355697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.422708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:08.428258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:08.449407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 678/1342] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:08.453994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.483959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e88c84-58eb-433b-9a41-c22190d3369d service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.497818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:08.499325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.500601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.501505 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.503166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:08.504509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:08.522020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:08.539276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 665/1343] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:08.543979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.556888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc2dea-1254-4794-ac6b-991382d0584c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.571540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:08.573660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.574145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.576060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.576904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:08.583040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:08.642305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.644423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:08.660762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12a053-3624-48eb-8f51-fd97053b4e50 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.667187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 679/1344] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:08 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:08.671527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.676219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:08.676973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.677253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.680390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] PATH_INFO: `/v3/projects/076d5555468141fb8835eb9dc9317671` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.683842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076d5555468141fb8835eb9dc9317671)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:08.689270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:08.777509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc94eed-7987-4d87-95e1-84c3db96fed7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.784193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:08.784832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.785111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.785386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:08.786241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:08.788310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:08.827665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076d5555468141fb8835eb9dc9317671', 'request_id': 'req-e4362b85-9626-4c54-88ef-0ce4a0335592'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:08.833986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076d5555468141fb8835eb9dc9317671', 'request_id': 'req-e4362b85-9626-4c54-88ef-0ce4a0335592'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:08.841443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076d5555468141fb8835eb9dc9317671', 'request_id': 'req-e4362b85-9626-4c54-88ef-0ce4a0335592'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:08.910439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 680/1345] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:08 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:08.912741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:08.934241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02192761-6d52-418f-8bec-46b7832e9615 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:08.938019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] The token cache is being invalidate because project 076d5555468141fb8835eb9dc9317671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:08.940971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:08.940971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e4362b85-9626-4c54-88ef-0ce4a0335592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:08.941535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 666/1346] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:08 2022] DELETE /identity/v3/projects/076d5555468141fb8835eb9dc9317671 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:08.947705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:08.948195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:08.948740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.001484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:09.003264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.038153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.038883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.041244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.041678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.042533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.044398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c775c8-1c1e-4650-8aa3-bb33ebf3dd8a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.063115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 681/1347] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:09.065946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.067000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.067497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.124038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] MFA Rules not processed for user `fbbd63f10220465c996e6c706f1ef4a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:09.126746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.152608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 667/1348] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:09.153282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.159786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.194028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382eea2-d02e-4ce0-a400-574005c32ad7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:09.205910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.206355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.206631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.206921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.207742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.209235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0535e7-87e9-4637-9d32-aaae4cd7d88f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.246091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 668/1349] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:09.247684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc59457-76d7-4bc9-bcba-2e96d7468911 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.249591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc59457-76d7-4bc9-bcba-2e96d7468911 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.251435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc59457-76d7-4bc9-bcba-2e96d7468911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.276747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-fbc59457-76d7-4bc9-bcba-2e96d7468911 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:09.278309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fbc59457-76d7-4bc9-bcba-2e96d7468911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.284882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 682/1350] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:09.285802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.295798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 669/1351] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:09.301627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.301627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.301627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.306746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3e827-70f8-4699-9a11-94919e16a457 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:09.316694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.318718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.319701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.320118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] PATH_INFO: `/v3/projects/0cf0ded643384150882993bf7c24f04a/users/f9723d031fce435b843352f7f288e180/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.355867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] MFA Rules not processed for user `fbbd63f10220465c996e6c706f1ef4a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:09.357105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.358363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf0ded643384150882993bf7c24f04a, user_id=f9723d031fce435b843352f7f288e180, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.365355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.380246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:09.383192 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 670/1352] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:09.383722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.404371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66fc97-f3d8-45ea-8bfb-1778db1b8142 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:09.424256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.425693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.426282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.426895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.428067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:09.428594 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.431409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.457191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 671/1353] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:09.459545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.463284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.480173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 683/1354] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:09 2022] PUT /identity/v3/projects/0cf0ded643384150882993bf7c24f04a/users/f9723d031fce435b843352f7f288e180/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:09.480173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.611687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e148f-8006-44bd-9173-833c54ed7c38 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:09.641355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825110e-2424-4c7b-9df6-05c1c0b3f72d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:09.654214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.654861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.655143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.655413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.656052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.658007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3584d0-e39a-43b1-9af0-3a21d22f42cc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.749636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:09.750890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.753304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.753829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.827023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 684/1355] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/projects => generated 367 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:09.828335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:09.829894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:09.830328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:09.863676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:09.866418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:09.892979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 672/1356] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:09.899420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:09.900248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] MFA Rules not processed for user `f9723d031fce435b843352f7f288e180`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:09.901736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:09.924045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ff374-81f3-4fb8-9919-6b9d54b26eb5 service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.093699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.102515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.102515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.102515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.102515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.102515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.108345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 685/1357] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:10.114750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.146263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af936e-511c-4cba-bd92-3f1a6d47539d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.165967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.167127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.179580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.179924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:10.196363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.199017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.205467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.269560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 673/1358] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:09 2022] POST /identity/v3/users => generated 284 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:10.271362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.306201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96878ed9-117a-48ae-9ee2-1affc90e6721 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.320467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.321112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.321389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.321660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.324174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.325895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.359442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 674/1359] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:10.370859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 686/1360] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:10 2022] POST /identity/v3/projects => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:10.372725 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.375015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.421498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37e6b3-9b99-4bcb-9e5a-a87f2829f6b9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.602076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a0bca-1a48-4546-99c8-df7be531be9e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.619730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.621845 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.622301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.622613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.627358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.628414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.629059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.629339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.629613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.630264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.631619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.632379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.666130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 675/1361] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:10.668761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.697884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f8b8-d378-44f5-8b3f-0730bba61ae9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.714292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.716442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.716677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.718537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] PATH_INFO: `/v3/projects/32e677a55715432fa699f9e22ec009a4/users/3c517125067a4229936a7400ad2f4996/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.757233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 687/1362] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:10 2022] POST /identity/v3/users => generated 298 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:10.761330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.777334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fd986-966a-42d9-b91a-90afa05de79b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.789906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e677a55715432fa699f9e22ec009a4, user_id=3c517125067a4229936a7400ad2f4996, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.790198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:10.790882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.791257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.791581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:10.792247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:10.793880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.795827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25822c17-7f24-4ef9-b46f-27439a2473ad admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:10.825788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 688/1363] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:10.826766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:10.868016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da34c1-054e-4719-9215-6884f6209b8d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:10.872876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 676/1364] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:10 2022] PUT /identity/v3/projects/32e677a55715432fa699f9e22ec009a4/users/3c517125067a4229936a7400ad2f4996/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:10.887772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:10.888113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:10.888320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.010104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] MFA Rules not processed for user `3c517125067a4229936a7400ad2f4996`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:11.018945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:11.035161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:11.052848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:11.053546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.053834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.054115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] PATH_INFO: `/v3/projects/627ba77bc0b64985acbea2da558c87e4/users/34a3467a5b9144b4851ac74135175d94/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.116341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=627ba77bc0b64985acbea2da558c87e4, user_id=34a3467a5b9144b4851ac74135175d94, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:11.139535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:11.251813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 677/1365] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:11.253347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:11.255493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:11.261986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 689/1366] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:10 2022] PUT /identity/v3/projects/627ba77bc0b64985acbea2da558c87e4/users/34a3467a5b9144b4851ac74135175d94/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:11.263675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:11.388854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db96260-4112-44c1-bc5e-3f160069eb94 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:11.409547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:11.410389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.410681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.410985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.412595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:11.412730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca29a7-975c-44cc-a96c-f0b08d17448c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:11.413696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:11.450295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 690/1367] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:11.451494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:11.478375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9125d-9b87-45be-b044-8426fc1dd315 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:11.507559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:11.507759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.507958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.510424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] PATH_INFO: `/v3/projects/627ba77bc0b64985acbea2da558c87e4/users/34a3467a5b9144b4851ac74135175d94/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.514392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=627ba77bc0b64985acbea2da558c87e4, user_id=34a3467a5b9144b4851ac74135175d94, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:11.523844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da383c-6c78-408c-98f5-0d9ba3feb55a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:11.576761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 691/1368] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:11 2022] PUT /identity/v3/projects/627ba77bc0b64985acbea2da558c87e4/users/34a3467a5b9144b4851ac74135175d94/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:11.576761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45f03-ec4c-47b3-8250-ad087702a8fb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.576761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45f03-ec4c-47b3-8250-ad087702a8fb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.576761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec45f03-ec4c-47b3-8250-ad087702a8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.594646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:11.597907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.598179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:11.598856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.599324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.653396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-3ec45f03-ec4c-47b3-8250-ad087702a8fb None None] MFA Rules not processed for user `34a3467a5b9144b4851ac74135175d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:11.656324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3ec45f03-ec4c-47b3-8250-ad087702a8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:11.818103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:11.819765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:11.857732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 678/1369] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:11.889813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 692/1370] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:11.913133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:11.916824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818bd56-8aea-47c0-8ab4-d62ba7c47e00 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.917115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818bd56-8aea-47c0-8ab4-d62ba7c47e00 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.917392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818bd56-8aea-47c0-8ab4-d62ba7c47e00 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.919547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:11.921024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 693/1371] 10.222.0.42 () {54 vars in 912 bytes} [Wed Oct 26 11:18:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:11.926926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae410-5833-4a5a-a676-040431b87a4c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:11.927213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae410-5833-4a5a-a676-040431b87a4c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:11.927486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ae410-5833-4a5a-a676-040431b87a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:11.932333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:11.949187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1269f-3cb0-4bb1-9a48-8121c6760e36 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:12.020428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-fa8ae410-5833-4a5a-a676-040431b87a4c None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:12.022061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fa8ae410-5833-4a5a-a676-040431b87a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:12.090320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:12.091007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:12.091291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:12.091566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:12.132152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:12.137978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:12.145366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:12.184170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 679/1372] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:12.201766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 694/1373] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:18:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:12.252661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:12.256022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:12.263281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:12.263739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:12.264239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:12.276311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd01d77-7420-42a3-b5df-cfea3cf437df service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:12.286149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:12.287009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:12.287476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:12.289182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:12.313310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:12.315368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:12.334379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 680/1374] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:12.352837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] MFA Rules not processed for user `e031dd9da25146eebb8813dc8bad8b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:12.353420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:12.551895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:12.565430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130a8e5-0c7f-4d05-a3bc-3df63327142d service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:12.582383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:12.583040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:12.583322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:12.583598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:12.596429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 695/1375] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:12.611431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:12.611431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:12.632167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:12.716239 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:12.717890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:12.741199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 681/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:12.773655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89919629-f866-49e2-b14e-35aafc5d0160 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:12.783511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:12.783979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:12.784267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:12.784812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:12.794238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:12.796126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886672-a1d7-4c5f-8dbe-003adb0fc46d service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:12.807207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 696/1377] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:12.826549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:14.117632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:14.123739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:14.273048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deb070-d562-4ef1-a48b-55b1bfbb4328 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:14.279754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:14.279754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:14.280143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:14.280535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:14.282693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:14.298704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:14.301026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62926e4a-ae0a-49ef-9629-1bf694f52e04 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:14.320144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 682/1378] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:16.040443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d7228-c896-4a07-88a0-3b76df965a0f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:16.040866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d7228-c896-4a07-88a0-3b76df965a0f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:16.040992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d7228-c896-4a07-88a0-3b76df965a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:16.088385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0e1d7228-c896-4a07-88a0-3b76df965a0f None None] MFA Rules not processed for user `41276c93e476436791b89ab8a0d4537d`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:16.091436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0e1d7228-c896-4a07-88a0-3b76df965a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:16.123975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:16.166857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:16.167959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:16.168370 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:16.224371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] MFA Rules not processed for user `ff0c5f8521534fc1a0a9c5ea28fbaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:16.225628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:16.227311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 697/1379] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:16.236607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:16.439690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 683/1380] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:16.450157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812438-c09e-4b83-b744-306c0e6bb578 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:16.450511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812438-c09e-4b83-b744-306c0e6bb578 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:16.450941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60812438-c09e-4b83-b744-306c0e6bb578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:16.472009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-60812438-c09e-4b83-b744-306c0e6bb578 None None] MFA Rules not processed for user `ff0c5f8521534fc1a0a9c5ea28fbaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:16.473519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-60812438-c09e-4b83-b744-306c0e6bb578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:16.484720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 698/1381] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:16.492477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:16.505435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5ec72-39bf-4981-99e0-1d0156fce1bb None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:16.511082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:16.511870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:16.512283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:16.512706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:16.521176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:16.522781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95f14c-80b9-4944-8637-2a28d79150c9 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:16.538992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 684/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:17.379934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:17.380477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:17.380598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:17.382427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 699/1383] 10.222.0.42 () {54 vars in 912 bytes} [Wed Oct 26 11:18:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:17.392601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad4a-aaed-4402-b0b9-b027150d77d9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:17.393169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad4a-aaed-4402-b0b9-b027150d77d9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:17.393401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317ad4a-aaed-4402-b0b9-b027150d77d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:17.420774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d317ad4a-aaed-4402-b0b9-b027150d77d9 None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:17.423388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d317ad4a-aaed-4402-b0b9-b027150d77d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:17.445159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 685/1384] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:17.465767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:17.473725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:17.497430 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a1493-c9f7-4f16-b2a4-019f7a1679cd None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:17.506658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:17.507846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:17.508365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:17.508866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:17.535956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:17.537897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19cdb35-d1a3-4f1d-8279-a3fa56e05721 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:17.552661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 700/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:17.665652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ffda-5bf8-40e3-a43f-ac3e27475bc3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:17.666547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ffda-5bf8-40e3-a43f-ac3e27475bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:17.666547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3ffda-5bf8-40e3-a43f-ac3e27475bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:17.717612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-85f3ffda-5bf8-40e3-a43f-ac3e27475bc3 None None] MFA Rules not processed for user `e031dd9da25146eebb8813dc8bad8b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:17.718887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-85f3ffda-5bf8-40e3-a43f-ac3e27475bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:17.738995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 686/1386] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:17.942608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:17.942608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:17.942608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:17.967600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] MFA Rules not processed for user `ff0c5f8521534fc1a0a9c5ea28fbaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:17.968819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:17.981880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 701/1387] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:19.566575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:19.567196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:19.567196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:19.795165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:19.798871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:19.971462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 687/1388] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:18:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:20.517044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:20.520235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:20.762694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcade24a-2d07-474d-ac6c-0e801d4d4c73 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:20.774807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:20.775575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:20.775922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:20.776264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:20.786379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:20.789795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:20.798603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 702/1389] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:18:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:18:21.757626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:21.765843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:21.938086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:21.972172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745726f-8bb4-49a2-937a-4bcd2233de52 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:21.993439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:22.004171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:22.004657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:22.008024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] PATH_INFO: `/v3/users/da9a0e67d59b49d4aa945b4da77ad2af` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:22.012495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9a0e67d59b49d4aa945b4da77ad2af)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:22.023187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:22.038199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:22.063748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b782153-3bea-4e7d-92f3-242d9d0088a9 service swift] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:22.101474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:22.101474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:22.101474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:22.101474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] PATH_INFO: `/v3/users/f9723d031fce435b843352f7f288e180` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:22.102106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9723d031fce435b843352f7f288e180)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:22.102106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:22.569724 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9723d031fce435b843352f7f288e180'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.575397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9a0e67d59b49d4aa945b4da77ad2af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.596660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9723d031fce435b843352f7f288e180'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.597533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9a0e67d59b49d4aa945b4da77ad2af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.597950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9723d031fce435b843352f7f288e180'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.598538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9a0e67d59b49d4aa945b4da77ad2af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.601927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:22.711135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9723d031fce435b843352f7f288e180'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.713143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ae281b92-d23c-4855-ac70-0d78996b33df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9723d031fce435b843352f7f288e180'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.756741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 703/1390] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:22 2022] DELETE /identity/v3/users/f9723d031fce435b843352f7f288e180 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:22.768095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17165463-d59b-486f-994b-91349af46623 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:22.768095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17165463-d59b-486f-994b-91349af46623 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:22.768095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17165463-d59b-486f-994b-91349af46623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:22.793921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-17165463-d59b-486f-994b-91349af46623 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:22.794789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-17165463-d59b-486f-994b-91349af46623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:22.813665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9a0e67d59b49d4aa945b4da77ad2af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.814936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4fe78d6a-6fd5-49a5-95cc-02cd4187bc21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9a0e67d59b49d4aa945b4da77ad2af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:22.869043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 688/1391] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:21 2022] DELETE /identity/v3/users/da9a0e67d59b49d4aa945b4da77ad2af => generated 0 bytes in 1112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:22.892718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:22.892718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:22.892718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:22.959694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:22.963074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:23.037797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 704/1392] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:23.049572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17165463-d59b-486f-994b-91349af46623 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:23.051505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-17165463-d59b-486f-994b-91349af46623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:23.153548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 689/1393] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:23.165237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:23.170095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:23.195057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17165463-d59b-486f-994b-91349af46623 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:23.364624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:23.374534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:23.374534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:23.374534 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:23.382704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:23.391624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee58b97-c82f-45ed-a609-ede8d90e6b7b service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:23.440408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 705/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:23.442296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:23.442651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:23.443028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:23.493288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:23.497834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:23.519631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 706/1395] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Oct 26 11:18:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:23.561307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:23.574032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6866db-a390-4a07-b11c-0a021ea908d9 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:23.588155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:23.589013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:23.589451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:23.589859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:23.598899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68b541-7dd3-42e5-b4f7-99b0ded223d9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:23.614376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:23.615042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:23.615319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:23.615590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:23.619425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:23.625959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:23.646338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:23.648311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:23.658935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 707/1396] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:23.675617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 690/1397] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:25.091164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:25.100122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:25.140319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1dacc-f35b-4fec-894b-59682ca5e1bb service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:25.152510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:25.153606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:25.154040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:25.154576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] PATH_INFO: `/v3/projects/7c0fb82767b3409098903b6ba8dc7b62` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:25.156119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0fb82767b3409098903b6ba8dc7b62)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:25.158941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:25.189979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:25.303502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:25.305449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:25.331063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a3f0b-fef6-4cb7-ad4a-5f27cf7deb1b service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:25.379076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:25.394025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:25.396087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:25.397128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0fb82767b3409098903b6ba8dc7b62', 'request_id': 'req-0e6a5a84-351c-4702-ad8d-f8440a52c080'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.398110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] PATH_INFO: `/v3/projects/0cf0ded643384150882993bf7c24f04a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:25.409322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf0ded643384150882993bf7c24f04a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:25.412068 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:25.418067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0fb82767b3409098903b6ba8dc7b62', 'request_id': 'req-0e6a5a84-351c-4702-ad8d-f8440a52c080'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.435459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0fb82767b3409098903b6ba8dc7b62', 'request_id': 'req-0e6a5a84-351c-4702-ad8d-f8440a52c080'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.487620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:25.554847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] The token cache is being invalidate because project 7c0fb82767b3409098903b6ba8dc7b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:25.555390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.556388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.558261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 708/1398] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:25 2022] DELETE /identity/v3/projects/7c0fb82767b3409098903b6ba8dc7b62 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:25.562542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:25.564143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:25.710065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf0ded643384150882993bf7c24f04a', 'request_id': 'req-4245e938-2c53-42e2-a0ef-14e34f53531b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.727546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a84-351c-4702-ad8d-f8440a52c080 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:25.738790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf0ded643384150882993bf7c24f04a', 'request_id': 'req-4245e938-2c53-42e2-a0ef-14e34f53531b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.791684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf0ded643384150882993bf7c24f04a', 'request_id': 'req-4245e938-2c53-42e2-a0ef-14e34f53531b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.840952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:25.842415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:25.842771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:25.843164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] PATH_INFO: `/v3/users/289041fab15d428db7cd9b7af57fa7ea` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:25.846192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289041fab15d428db7cd9b7af57fa7ea)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:25.848650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:25.911479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] The token cache is being invalidate because project 0cf0ded643384150882993bf7c24f04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:25.912104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.917363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:25.919350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 691/1399] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:25 2022] DELETE /identity/v3/projects/0cf0ded643384150882993bf7c24f04a => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:25.923532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:25.928431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:25.951749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245e938-2c53-42e2-a0ef-14e34f53531b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:25.966414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:25.967398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:25.967754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:25.968104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] PATH_INFO: `/v3/users/34a3467a5b9144b4851ac74135175d94` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:25.971101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a3467a5b9144b4851ac74135175d94)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:25.973250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:26.115771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289041fab15d428db7cd9b7af57fa7ea'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.118338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:26.126981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289041fab15d428db7cd9b7af57fa7ea'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.129211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289041fab15d428db7cd9b7af57fa7ea'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.239450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289041fab15d428db7cd9b7af57fa7ea'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.240443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289041fab15d428db7cd9b7af57fa7ea'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.283267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 709/1400] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:25 2022] DELETE /identity/v3/users/289041fab15d428db7cd9b7af57fa7ea => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:26.285021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:26.300276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:26.329775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a3467a5b9144b4851ac74135175d94'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.375560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a3467a5b9144b4851ac74135175d94'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.376473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a3467a5b9144b4851ac74135175d94'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.494562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a3467a5b9144b4851ac74135175d94'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.495457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2afcd181-756c-4f4c-9453-59aad49b0618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a3467a5b9144b4851ac74135175d94'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.496177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3aedb-a9aa-4751-a704-3c8bc4a87293 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:26.504147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:26.505055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:26.505410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:26.506731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] PATH_INFO: `/v3/users/cc6727ff272f47c8b30faf741e108e52` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:26.508595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6727ff272f47c8b30faf741e108e52)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:26.511527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:26.556240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 692/1401] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:25 2022] DELETE /identity/v3/users/34a3467a5b9144b4851ac74135175d94 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:26.808359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6727ff272f47c8b30faf741e108e52'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.830068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6727ff272f47c8b30faf741e108e52'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.830934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6727ff272f47c8b30faf741e108e52'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.919909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6727ff272f47c8b30faf741e108e52'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.923888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6727ff272f47c8b30faf741e108e52'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:26.974336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 710/1402] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:26 2022] DELETE /identity/v3/users/cc6727ff272f47c8b30faf741e108e52 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:26.979874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a2c-f576-41b9-8929-ed3822f2eda3 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:26.980162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a2c-f576-41b9-8929-ed3822f2eda3 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:26.980587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f8a2c-f576-41b9-8929-ed3822f2eda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:27.012665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3c4f8a2c-f576-41b9-8929-ed3822f2eda3 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:27.014339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3c4f8a2c-f576-41b9-8929-ed3822f2eda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:27.259176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 693/1403] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:27.324566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:27.326210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:27.439432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:27.461987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad01b-7941-4d3c-892b-2e481029bd16 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:27.470726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:27.471793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:27.472333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:27.473019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:27.489802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:27.492652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:27.511333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 711/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:28.252406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71e9c-6521-4e82-bc08-5a88115c206c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:28.253018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71e9c-6521-4e82-bc08-5a88115c206c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:28.253018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af71e9c-6521-4e82-bc08-5a88115c206c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:28.254363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 694/1405] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:28.272175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:28.273839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:28.289424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735ea0-de60-462c-a0b0-13ceb8687d3c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:28.289710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735ea0-de60-462c-a0b0-13ceb8687d3c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:28.289988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735ea0-de60-462c-a0b0-13ceb8687d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:28.323674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547371e-66be-4313-975d-1525d84d9a63 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:28.346099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:28.346908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:28.347069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:28.347300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] PATH_INFO: `/v3/projects/5e624ae9521e4b8191d8a86bff142951` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:28.350888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e624ae9521e4b8191d8a86bff142951)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:28.355712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:28.355837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-79735ea0-de60-462c-a0b0-13ceb8687d3c None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:28.356439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-79735ea0-de60-462c-a0b0-13ceb8687d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:28.373966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:28.555002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 695/1406] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:28.567477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e624ae9521e4b8191d8a86bff142951', 'request_id': 'req-61ea396d-0061-4ae3-b78e-155de58e608e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:28.584436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e624ae9521e4b8191d8a86bff142951', 'request_id': 'req-61ea396d-0061-4ae3-b78e-155de58e608e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:28.615980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e624ae9521e4b8191d8a86bff142951', 'request_id': 'req-61ea396d-0061-4ae3-b78e-155de58e608e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:28.712967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] The token cache is being invalidate because project 5e624ae9521e4b8191d8a86bff142951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:28.712967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:28.715865 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:28.719687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 712/1407] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:28 2022] DELETE /identity/v3/projects/5e624ae9521e4b8191d8a86bff142951 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:28.722041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e300b-44f6-4d90-a734-23bd17433a79 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:28.722424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e300b-44f6-4d90-a734-23bd17433a79 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:28.722795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e300b-44f6-4d90-a734-23bd17433a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:28.766645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:28.768188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:28.785325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5f8e300b-44f6-4d90-a734-23bd17433a79 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:28.787793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5f8e300b-44f6-4d90-a734-23bd17433a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:28.974668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 696/1408] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:28.982503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:28.984050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:28.986684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:28.992324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea396d-0061-4ae3-b78e-155de58e608e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.009482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.010266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.010610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.010969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.019660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:29.033374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:29.037459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:29.043703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.045573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.065453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 713/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:29.083302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 697/1410] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:29.085197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.113706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf54d5-7d6b-40c6-ab20-2cb2ff0c5d07 service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.128604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.129732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.130858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.131131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.131789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:29.132088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.133718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.195736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 714/1411] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:29.196529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.211710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953efc94-2af8-4805-bfca-0657f2db799a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.237292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.237935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.238214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.238491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.239199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.241644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.407273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 698/1412] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:29 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:29.412166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.435909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da86c1a-18d5-4d47-a8cb-4ccc7e5c1e23 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.577625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:29.609804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.612256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.612256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.612256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.612256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.615355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.690179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.692142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:29.712403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae03db1-6f17-4a14-a214-4726fcfe6134 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.718366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.719650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.728499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.731970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] PATH_INFO: `/v3/projects/627ba77bc0b64985acbea2da558c87e4` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.731970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=627ba77bc0b64985acbea2da558c87e4)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.732341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.769959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 715/1413] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:29 2022] POST /identity/v3/users => generated 283 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:29.781117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.797635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65709d4b-e204-4073-b13b-485b5b8fc1e0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.828730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:29.830882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:29.830882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:29.830882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:29.835931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:29.838195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:29.874320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 716/1414] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:29.877039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:29.907850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf94149-ead1-41c4-8f81-3352c44dfe7e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:29.922928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '627ba77bc0b64985acbea2da558c87e4', 'request_id': 'req-136864cb-b924-40e8-a50d-d9ceaf462b6e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:29.936966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '627ba77bc0b64985acbea2da558c87e4', 'request_id': 'req-136864cb-b924-40e8-a50d-d9ceaf462b6e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:29.960395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '627ba77bc0b64985acbea2da558c87e4', 'request_id': 'req-136864cb-b924-40e8-a50d-d9ceaf462b6e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:30.051407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.052640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.052934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.053214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.053866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.055518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:30.075252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 717/1415] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:30.083525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.083674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:30.083904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] The token cache is being invalidate because project 627ba77bc0b64985acbea2da558c87e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:30.084358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:30.085453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-136864cb-b924-40e8-a50d-d9ceaf462b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:30.087231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 699/1416] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:29 2022] DELETE /identity/v3/projects/627ba77bc0b64985acbea2da558c87e4 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:30.092802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d27b-5be0-465d-94a2-e5b67fff1dcb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.093442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d27b-5be0-465d-94a2-e5b67fff1dcb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.094036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82d27b-5be0-465d-94a2-e5b67fff1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.109152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924db64-e166-4022-921d-ac6ec14fcba1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.130045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.131927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.131927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.132288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] PATH_INFO: `/v3/projects/aa0c4336b23b4ce7a7000e2824ce2308/users/1817bebc03014f0aa9e8468684ea1918/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.154136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0c4336b23b4ce7a7000e2824ce2308, user_id=1817bebc03014f0aa9e8468684ea1918, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.155199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0e82d27b-5be0-465d-94a2-e5b67fff1dcb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:30.157516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200ae84-52f5-472a-9b0d-fdd71dab749e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:30.174072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0e82d27b-5be0-465d-94a2-e5b67fff1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:30.191611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 700/1417] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:30.194503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.195229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.200910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.254092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:30.258021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:30.288660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 718/1418] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:30 2022] PUT /identity/v3/projects/aa0c4336b23b4ce7a7000e2824ce2308/users/1817bebc03014f0aa9e8468684ea1918/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:30.303574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.304430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 701/1419] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:30.305929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.307091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.309388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.341726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adfa48-85d7-4642-b287-ce7f867e2d3b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.378797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] MFA Rules not processed for user `1817bebc03014f0aa9e8468684ea1918`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:30.380929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:30.487721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.489083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.489566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.491701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.492937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:30.493423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.496280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:30.535967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 702/1420] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:30.540078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.574691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9325c09-53ed-4fba-9673-ac410e448b62 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.588769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 719/1421] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:30.590316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.591829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.592652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.592652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.593623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.595304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.596588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:30.600856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:30.614419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3522817-7a03-4449-b2b8-7325ca649b9f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.621401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.622038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.622449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.623509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.625846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.629740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:30.774454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:30.776861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 720/1422] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/projects => generated 363 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:30.777904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.786590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 703/1423] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:30.788793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:30.792316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:30.929030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d473548-167d-4047-b492-7643f899cc9c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.967901 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c683fe-230d-4d16-bf04-1c08a5884739 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:30.975067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:30.978742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:30.979125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:30.979576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:30.980201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:30.987840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.030388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.030388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.030388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.030388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] PATH_INFO: `/v3/projects/0791e18214d94c2495a75a53ed80283f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.032139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0791e18214d94c2495a75a53ed80283f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.034691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.086622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 704/1424] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:30 2022] POST /identity/v3/users => generated 288 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:31.087488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.114834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d278-8ffd-4b17-8154-29c6a956c432 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:31.129418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.131209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.131706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.132290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.134855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.139210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cc9ff-b91a-43a4-a8a4-41605dced910 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.216106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 705/1425] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:31 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:31.217818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af6ff-516a-4ec5-8d94-d0a4225d8870 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.219243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af6ff-516a-4ec5-8d94-d0a4225d8870 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.220001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af6ff-516a-4ec5-8d94-d0a4225d8870 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.225618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0791e18214d94c2495a75a53ed80283f', 'request_id': 'req-a6da694b-e2e4-4554-ac5e-66d17932e82f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.226945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 706/1426] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:18:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:18:31.228710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1af6ff-516a-4ec5-8d94-d0a4225d8870 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.245511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0791e18214d94c2495a75a53ed80283f', 'request_id': 'req-a6da694b-e2e4-4554-ac5e-66d17932e82f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.275682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0791e18214d94c2495a75a53ed80283f', 'request_id': 'req-a6da694b-e2e4-4554-ac5e-66d17932e82f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.307849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:31.395405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] The token cache is being invalidate because project 0791e18214d94c2495a75a53ed80283f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:31.395927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.398907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.402108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 721/1427] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:30 2022] DELETE /identity/v3/projects/0791e18214d94c2495a75a53ed80283f => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:31.403831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.406992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:31.419376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1af6ff-516a-4ec5-8d94-d0a4225d8870 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:31.442951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.447666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.447666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.447666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.447666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.447948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da694b-e2e4-4554-ac5e-66d17932e82f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:31.452748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaed2ea-382a-4bf3-93bd-f40c275a6fc3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.461791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.463047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.465830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.470710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.470710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.472512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.492938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 707/1428] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:31.499158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.499842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.500898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.504016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 722/1429] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:31.504016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.509777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:31.531178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e19115-546f-4607-8040-969c3201815f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:31.538271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.539321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.539657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.539917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] PATH_INFO: `/v3/users/4c491023f23142baa9e7531e4e409674` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.541293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c491023f23142baa9e7531e4e409674)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.549863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.561193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:31.564640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:31.622547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:31.750709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 708/1430] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:18:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:31.767290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.771117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:31.814198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf1ce15-3a3d-4f00-bfa3-b87213485eb6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:31.826887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:31.827953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:31.828321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:31.828809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] PATH_INFO: `/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3/users/7309d89f76e14e2194b7d028f244caf6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:31.836965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c491023f23142baa9e7531e4e409674'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.872801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c491023f23142baa9e7531e4e409674'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.873367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6168812d6d2049de91f7ad0a1a91c0f3, user_id=7309d89f76e14e2194b7d028f244caf6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:31.874858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c491023f23142baa9e7531e4e409674'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.877954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:31.979093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 709/1431] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:31 2022] PUT /identity/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3/users/7309d89f76e14e2194b7d028f244caf6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:31.980077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:31.988260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c491023f23142baa9e7531e4e409674'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:31.989219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c491023f23142baa9e7531e4e409674'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:32.033878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 723/1432] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:31 2022] DELETE /identity/v3/users/4c491023f23142baa9e7531e4e409674 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:32.036396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:32.039031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.103294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253fa4b-411e-40a4-9523-5329e18da92f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:32.180853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806aa65-acef-429a-a9a4-2046ad286642 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:32.190285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:32.193933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:32.194723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.195178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.195634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.196379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:32.198206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:32.212519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 724/1433] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:32.215730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:32.221129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.277994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:32.278958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.279445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.280221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.285011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:32.285011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:32.298323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 710/1434] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:32.299362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:32.313331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0735ca3-3b95-4334-a03e-94956c50cc3d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:32.335924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:32.336884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.337093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.342496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] PATH_INFO: `/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3/users/7309d89f76e14e2194b7d028f244caf6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.342496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6168812d6d2049de91f7ad0a1a91c0f3, user_id=7309d89f76e14e2194b7d028f244caf6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:32.343544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:32.400120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 711/1435] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:32 2022] PUT /identity/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3/users/7309d89f76e14e2194b7d028f244caf6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:32.402395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:32.408274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.439673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df32c6a-02c8-4fae-aa50-a744cab8052d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:32.581732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd962a09-bb8e-4309-8242-f9308f8fdd6b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:32.590221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:32.591064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.591390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.591668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] PATH_INFO: `/v3/projects/c4f057b686054bcfb786edc7136c89a2/users/41c9ef5de3d14dca94541d5786358789/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.618443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f057b686054bcfb786edc7136c89a2, user_id=41c9ef5de3d14dca94541d5786358789, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:32.622097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419713c-ce18-4e51-b2e5-930cbd09baaf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:32.628953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:32.629801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.630193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.630584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.633382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.649223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:32.649394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00bb89-544a-442d-9c97-575a59b99bd2 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:32.673980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 725/1436] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:32.675761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.677416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.678401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.710613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 712/1437] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:32 2022] PUT /identity/v3/projects/c4f057b686054bcfb786edc7136c89a2/users/41c9ef5de3d14dca94541d5786358789/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:32.720002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:32.720329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:32.720688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:32.762503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] MFA Rules not processed for user `7309d89f76e14e2194b7d028f244caf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:32.764768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.800607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] MFA Rules not processed for user `41c9ef5de3d14dca94541d5786358789`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:32.802849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:32.976094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 726/1438] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:32.986766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:32.991784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:33.026201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 713/1439] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:33.030780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.172621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b83ef7-87ad-406e-93a6-f87366f744ca None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.172800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95aee1b-519e-447c-bf73-ca2e393d5285 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.172918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:33.172918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:33.172918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:33.172918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:33.173279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:33.174106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:33.175046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:33.175836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:33.176764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:33.177149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:33.197017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:33.202438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:33.231523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 727/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:33.301116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 714/1441] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:33 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:33.305968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.332070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6355dab-56cc-4784-87ca-c05e04f743f1 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.530473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:33.551264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:33.551580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:33.551862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:33.552637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:33.556988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:33.705117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 728/1442] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:33 2022] POST /identity/v3/users => generated 289 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:33.708748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.732867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13c454-973f-4e1d-8606-0a3de867b08c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.738301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:33.739109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:33.744840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:33.744840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:33.744840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:33.744840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:33.755304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 715/1443] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:33.759890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.772291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994eef03-d987-47c7-af51-1df9b9b48681 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.777657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:33.778359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:33.778638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:33.778994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] PATH_INFO: `/v3/projects/037f21b1cc394b9bbb36555a12025a7a/users/9392335b781c4877b8bba258b4d17bb7/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:33.803916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=037f21b1cc394b9bbb36555a12025a7a, user_id=9392335b781c4877b8bba258b4d17bb7, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:33.806915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:33.892411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.893442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 729/1444] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:33 2022] PUT /identity/v3/projects/037f21b1cc394b9bbb36555a12025a7a/users/9392335b781c4877b8bba258b4d17bb7/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:33.906957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca541ab-7f5d-4675-9d7d-f611f34d929e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:33.992355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:33.996993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:34.090528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:34.091338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.091723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.092208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.092995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:34.094969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:34.136766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 716/1445] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:34.138135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:34.166229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c66f6-d3ad-4073-9a21-434a2ec3e663 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:34.183016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:34.184211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.184737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.185202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] PATH_INFO: `/v3/projects/037f21b1cc394b9bbb36555a12025a7a/users/9392335b781c4877b8bba258b4d17bb7/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.191761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=037f21b1cc394b9bbb36555a12025a7a, user_id=9392335b781c4877b8bba258b4d17bb7, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:34.197812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f41c8a-ab97-4edd-abee-ed6d8753da0a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:34.218741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a29e75-31ee-488f-b5af-892f183fd0b8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:34.232732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 717/1446] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:34 2022] PUT /identity/v3/projects/037f21b1cc394b9bbb36555a12025a7a/users/9392335b781c4877b8bba258b4d17bb7/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:34.253892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.254340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.254689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.312714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] MFA Rules not processed for user `9392335b781c4877b8bba258b4d17bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:34.315958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:34.356049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:34.356760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.357069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.357578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] PATH_INFO: `/v3/users/3c517125067a4229936a7400ad2f4996` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.358837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c517125067a4229936a7400ad2f4996)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:34.360858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:34.505173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:34.525974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 718/1447] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:34.542269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:34.543658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:34.640853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193acfcf-936d-4397-8c21-d3481d8dd436 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:34.646038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:34.646804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.647783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.648121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.658397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c517125067a4229936a7400ad2f4996'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.676255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c517125067a4229936a7400ad2f4996'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.677282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c517125067a4229936a7400ad2f4996'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.682048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:34.684337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:34.694294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:34.717219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 719/1448] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:34.719272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:34.741718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:34.780083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a176d6-9cb3-46af-bb9a-a01959c9c4f5 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:34.785683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:34.790606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.790606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.790606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] PATH_INFO: `/v3/projects/c4d92ee9add34bdda4c6df36abe7c787` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.790606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d92ee9add34bdda4c6df36abe7c787)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:34.791094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:34.808289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c517125067a4229936a7400ad2f4996'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.808967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2cec786e-2549-4d76-994d-160c3b60d760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c517125067a4229936a7400ad2f4996'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.854732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 730/1449] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:33 2022] DELETE /identity/v3/users/3c517125067a4229936a7400ad2f4996 => generated 0 bytes in 860 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:34.856669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:34.856956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:34.857227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:34.901696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:34.903341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:34.925079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d92ee9add34bdda4c6df36abe7c787', 'request_id': 'req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.956208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d92ee9add34bdda4c6df36abe7c787', 'request_id': 'req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:34.964409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d92ee9add34bdda4c6df36abe7c787', 'request_id': 'req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.009899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 731/1450] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:35.011149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.026857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.038967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f836c9-c9be-4375-b4cf-bd555bd5accb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.044567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.045354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.045697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.046038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] PATH_INFO: `/v3/users/1817bebc03014f0aa9e8468684ea1918` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.047480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1817bebc03014f0aa9e8468684ea1918)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.049377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.055417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] The token cache is being invalidate because project c4d92ee9add34bdda4c6df36abe7c787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:35.055914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.056795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.058858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 720/1451] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:34 2022] DELETE /identity/v3/projects/c4d92ee9add34bdda4c6df36abe7c787 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:35.059815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.061567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.122944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7b2ea-6192-4818-a4ab-271b4b2baa2b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.126507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.127168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.127446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.127716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.129452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.141671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.143140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.153747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 721/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:35.154434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.155715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.226275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2405fd-5377-493e-b3b5-2bc84f79d1be service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.229960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.230642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.230967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.231244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.233039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.264251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1817bebc03014f0aa9e8468684ea1918'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.275756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1817bebc03014f0aa9e8468684ea1918'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.276701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1817bebc03014f0aa9e8468684ea1918'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.314377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.315859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0a3a-6826-47d9-b0d0-00994e20c54d service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.325352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 722/1453] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:35.326567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1827-6d35-4c04-8d9d-64a4d3e83a31 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.326947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1827-6d35-4c04-8d9d-64a4d3e83a31 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.327221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e1827-6d35-4c04-8d9d-64a4d3e83a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.348528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4e6e1827-6d35-4c04-8d9d-64a4d3e83a31 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:35.349742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4e6e1827-6d35-4c04-8d9d-64a4d3e83a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.359028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 723/1454] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:35.368390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.376278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.376278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.393609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1817bebc03014f0aa9e8468684ea1918'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.394308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-71a768b8-3d24-4cec-9b02-dbe9b4b8585a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1817bebc03014f0aa9e8468684ea1918'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:35.401215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:35.402473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.430371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 732/1455] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:35 2022] DELETE /identity/v3/users/1817bebc03014f0aa9e8468684ea1918 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:35.431742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 724/1456] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:35.439184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.442594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.443092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.448652 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.453055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71229d8-9a21-4f28-95bf-f81e704062a8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.500083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:35.501395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.534690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.535510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.535800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.536073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.536780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:35.537087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.539192 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.553085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 725/1457] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:35.557999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.571149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed7e89-9853-4f30-9dc0-4d7313468d01 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.576518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.577266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.577692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.578382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.579195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.582157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.604103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 733/1458] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:35.613997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.616306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.683578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d4c16-f186-4f0c-8514-b6c63adce314 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.688256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.689052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.689495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.689933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.703871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.708355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.717565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 734/1459] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:35.736098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.745999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:35.764185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 726/1460] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:35 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:35.768120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:35.775881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb59fe-ee5c-4250-8636-f1cb8516097e service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.898682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f0bff-109a-4e5e-bcfa-ff7e548cb900 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:35.905271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.907318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.907318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.907318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.907318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.909198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:35.914424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:35.916895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:35.916895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:35.916895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] PATH_INFO: `/v3/projects/32e677a55715432fa699f9e22ec009a4` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:35.918092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e677a55715432fa699f9e22ec009a4)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:35.920621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.017903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 727/1461] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:35 2022] POST /identity/v3/users => generated 294 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:36.019292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.033880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05ce97-3dbc-44a9-93c6-afee5235d63f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.041268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.042218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.043502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.043874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.045303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.047385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.064969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 728/1462] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:36.065515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.085287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab60a8-8704-4cb7-bd2d-47d3b6d525d8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.142672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e677a55715432fa699f9e22ec009a4', 'request_id': 'req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.152135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e677a55715432fa699f9e22ec009a4', 'request_id': 'req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.162948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e677a55715432fa699f9e22ec009a4', 'request_id': 'req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.255364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] The token cache is being invalidate because project 32e677a55715432fa699f9e22ec009a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:36.255703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.263977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f6dce694-7fa4-44f1-aa2c-ac962a4fee3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.271905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 735/1463] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:35 2022] DELETE /identity/v3/projects/32e677a55715432fa699f9e22ec009a4 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:36.277051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040d27-f92c-4a11-aa3b-3eb0669e5642 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.278438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040d27-f92c-4a11-aa3b-3eb0669e5642 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.278895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040d27-f92c-4a11-aa3b-3eb0669e5642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.292714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.297338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.297338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.297338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.297338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.299387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.320287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:36.337074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bd040d27-f92c-4a11-aa3b-3eb0669e5642 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:36.339421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bd040d27-f92c-4a11-aa3b-3eb0669e5642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:36.339748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 729/1464] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:36.341340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.343569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:36.361042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b559550-4f60-40b2-bf75-e762eb483887 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.363742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 736/1465] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:36.370744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.371828 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.372166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.372267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.372511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.372620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] PATH_INFO: `/v3/projects/9d2df5f568ea42fcbe400803c83ee1fe/users/9ae331bc8d2449619c782347a2e65c8b/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.372791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.391968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2df5f568ea42fcbe400803c83ee1fe, user_id=9ae331bc8d2449619c782347a2e65c8b, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.395051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91ac89-6571-457e-8ea9-cdd196084160 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.410095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:36.412429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:36.426299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 737/1466] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:36.438691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.454239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf94e93-c756-4f45-b0f1-2cc0f3605c05 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.464620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.465764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.466109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.466455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.467282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:36.467685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.469593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.484308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 738/1467] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:18:36.488984 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.500143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591a40e-cb6c-4113-9cfd-0b71df3546e3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.506529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.506529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.507277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.507585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.508227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.510138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.544377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 730/1468] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:36 2022] PUT /identity/v3/projects/9d2df5f568ea42fcbe400803c83ee1fe/users/9ae331bc8d2449619c782347a2e65c8b/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:36.555098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.555385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.556019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.584567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] MFA Rules not processed for user `9ae331bc8d2449619c782347a2e65c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:36.586281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:36.648532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 731/1469] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:36.652907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.668331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 739/1470] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:36.669642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.672274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:36.741472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5962b0-3c90-459d-99f1-6ec729356a4c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.768965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f340d-5241-4d41-beb4-96f08d77fe2a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:36.774492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.775246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.775721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.776471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] PATH_INFO: `/v3/projects/aa0c4336b23b4ce7a7000e2824ce2308` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.778904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:36.780548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0c4336b23b4ce7a7000e2824ce2308)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.782699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.813035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:36.813772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:36.814066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:36.814328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:36.815188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:36.817442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:36.869532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0c4336b23b4ce7a7000e2824ce2308', 'request_id': 'req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.875858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0c4336b23b4ce7a7000e2824ce2308', 'request_id': 'req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.881708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0c4336b23b4ce7a7000e2824ce2308', 'request_id': 'req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.940893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 732/1471] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/projects => generated 387 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:36.940893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.964857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] The token cache is being invalidate because project aa0c4336b23b4ce7a7000e2824ce2308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:36.969449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.969449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:36.969449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 740/1472] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:36 2022] DELETE /identity/v3/projects/aa0c4336b23b4ce7a7000e2824ce2308 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:36.969449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:36.969449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.002641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307978f1-99c6-47b4-84c9-53fe1b3040b8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.007527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.008136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.008424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.008701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.009360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.011626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.031674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fcaab-2dc1-4589-b790-6c6f2d90ad4f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.036211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.036808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.037086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.037354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.038539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.041920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.103810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 733/1473] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:37.104781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.106509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.119233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae59f3-8da8-4180-8571-51c97231961b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.121641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 741/1474] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:36 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:37.121641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.123689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.124321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.124943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.125225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.125499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] PATH_INFO: `/v3/users/7309d89f76e14e2194b7d028f244caf6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.126622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7309d89f76e14e2194b7d028f244caf6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.128563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.135006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b2a2-d3e1-4537-aca7-ea377efe8d27 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.139585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.140199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.140477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.140748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.141419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.143017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.153762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 742/1475] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:37.154566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.164846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66032376-d4e5-475c-9b48-da93155d40b9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.167725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:37.171158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.171719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.172013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.172307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.172924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.174522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.184684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 743/1476] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:37.185443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.194784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a69b5f-2a2a-4831-a547-146a1642f238 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.199335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.202172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.202172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.202172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.202172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.202563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.212105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 744/1477] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:37.213001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.223694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029b2f1-4234-4457-a3d7-67d62a5be589 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.228868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.229420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.229694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.229962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.230689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.232366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.237547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:37.246540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 745/1478] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:37.246540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.253528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcb95e-5972-4ff9-b749-b8c2a2b3e6a9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.258473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.259171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.259444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.259711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] PATH_INFO: `/v3/projects/d14eecab70af421cbf506d397c9fca4c/users/6206de16aa3145f7a03c605f48be3a96/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.275036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14eecab70af421cbf506d397c9fca4c, user_id=6206de16aa3145f7a03c605f48be3a96, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.278385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.292567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7309d89f76e14e2194b7d028f244caf6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:37.299308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7309d89f76e14e2194b7d028f244caf6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:37.299999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7309d89f76e14e2194b7d028f244caf6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:37.363757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 746/1479] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:37 2022] PUT /identity/v3/projects/d14eecab70af421cbf506d397c9fca4c/users/6206de16aa3145f7a03c605f48be3a96/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:37.368305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.393279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7309d89f76e14e2194b7d028f244caf6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:37.394070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-89625e16-fad5-4d6e-896e-9fb7adc0abf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7309d89f76e14e2194b7d028f244caf6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:37.420084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 734/1480] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:37 2022] DELETE /identity/v3/users/7309d89f76e14e2194b7d028f244caf6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:37.424551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.424840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.425117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.439328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d15a9b-268b-4d7f-9369-a4550f93c749 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.461231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] MFA Rules not processed for user `6206de16aa3145f7a03c605f48be3a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:37.463012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.510784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.511751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.512036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.512440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] PATH_INFO: `/v3/projects/22f753cd06384ef4b71a355e648de6f3/users/3c828fd5ff754beaaa34617c669d11ae/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.528873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f753cd06384ef4b71a355e648de6f3, user_id=3c828fd5ff754beaaa34617c669d11ae, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.531674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c1a14-6ae5-49af-8683-47beb9c6e27e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.557649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 735/1481] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:37.562103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.563476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.576886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c6641-6642-45bb-9928-b8d4784f5acf None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.581447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.582060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.582323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.582591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.583675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.587424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.610955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 747/1482] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:37 2022] PUT /identity/v3/projects/22f753cd06384ef4b71a355e648de6f3/users/3c828fd5ff754beaaa34617c669d11ae/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:37.623013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.623424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.624334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.654703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] MFA Rules not processed for user `3c828fd5ff754beaaa34617c669d11ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:37.657621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.721457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 736/1483] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:37 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:37.729684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.737061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 748/1484] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:37.741304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.742640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:37.834228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8d049-61dd-4270-9140-7fb1bce96599 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.840160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.840873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.841230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.841518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.842448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.842553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd4bf-1a39-411e-9fb0-7e2702ef3cd2 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.844122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.847983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:37.848373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:37.848655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:37.852985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:37.852985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:37.852985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:37.958855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 737/1485] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:37 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:37.962537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.982201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 749/1486] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:37 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:37.982702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cde5d-7a27-44a9-a117-f362fd754774 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:37.982802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:37.983793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.059280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31271e94-e250-4ff1-a70c-bfcba6f136de admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.064076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.064683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.064924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.065199 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] PATH_INFO: `/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.066154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.066331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6168812d6d2049de91f7ad0a1a91c0f3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.066849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.067110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.067384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.068041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.068230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.069622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.081347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 738/1487] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:38.081822 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.092461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ad0bd-a472-4ce3-941f-6bb89a9fad60 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.096922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.097509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.097791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.098061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.098750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.100684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.177031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6168812d6d2049de91f7ad0a1a91c0f3', 'request_id': 'req-8704444b-ed94-4fd6-ab32-fb93cb2f192a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:38.184003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6168812d6d2049de91f7ad0a1a91c0f3', 'request_id': 'req-8704444b-ed94-4fd6-ab32-fb93cb2f192a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:38.189440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 739/1488] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:38 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:38.190021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6168812d6d2049de91f7ad0a1a91c0f3', 'request_id': 'req-8704444b-ed94-4fd6-ab32-fb93cb2f192a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:38.190378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.200785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f917c1-7908-4908-8aa6-87af430c0942 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.239390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] The token cache is being invalidate because project 6168812d6d2049de91f7ad0a1a91c0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:38.239677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:38.240320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:38.241879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 750/1489] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:37 2022] DELETE /identity/v3/projects/6168812d6d2049de91f7ad0a1a91c0f3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:38.242718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.244685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.257014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.257683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.257961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.258232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] PATH_INFO: `/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b/users/3158691568524f2d89cb57c6aee6c372/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.274525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca0b2b6c22a4075b69efcdf04c0409b, user_id=3158691568524f2d89cb57c6aee6c372, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.277338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36625204-6f37-474f-9d14-c46c37991734 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.308650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704444b-ed94-4fd6-ab32-fb93cb2f192a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.313750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.314408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.314631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.314951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.315618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.317350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da94f3-8661-4a14-a5fa-5f070eab4bc4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.328270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 751/1490] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:38.329464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.329741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.330010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.349555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:38.350712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.359460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 752/1491] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:38.360314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.370119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 740/1492] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:38 2022] PUT /identity/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b/users/3158691568524f2d89cb57c6aee6c372/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:38.370801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16713ac-2dea-4d3a-80be-9e22ecc12caa None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.371536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.371917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.372328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.391080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:38.394670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:38.395911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.435191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:38.439622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.443104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.443104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.443104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] PATH_INFO: `/v3/projects/05a0bb9574144fa4bd2ecc612d00b284/users/a5c6e38b8bb74c4ba4714df0613f6141/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.455625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a0bb9574144fa4bd2ecc612d00b284, user_id=a5c6e38b8bb74c4ba4714df0613f6141, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.458244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc449c0-e3c8-4ecf-8d1f-9b833aa7273b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.460469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 741/1493] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:38.461529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.462655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.474297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e672dd-68f4-4cbb-95af-866392b7fe1a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.479639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.480234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.480511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.480776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.481405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.483008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.494460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 742/1494] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:38.495159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.504692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68985f57-6a9b-400e-9fad-63a650c763ff admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.509302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.511982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.511982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.511982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.511982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:38.511982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.512999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.525159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 743/1495] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:38.525791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.537889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91124a-5470-4257-af7b-6015aa3fe306 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.543880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.544724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.545002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.545269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] PATH_INFO: `/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b/users/3158691568524f2d89cb57c6aee6c372/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.547233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca0b2b6c22a4075b69efcdf04c0409b, user_id=3158691568524f2d89cb57c6aee6c372, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.549899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.560908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 744/1496] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:38 2022] PUT /identity/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b/users/3158691568524f2d89cb57c6aee6c372/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:38.561916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.583311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 753/1497] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:38 2022] PUT /identity/v3/projects/05a0bb9574144fa4bd2ecc612d00b284/users/a5c6e38b8bb74c4ba4714df0613f6141/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:38.584784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.585069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.585340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.616558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] MFA Rules not processed for user `3158691568524f2d89cb57c6aee6c372`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:38.617772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.643599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11475e-5dfb-4719-b4ee-a23014881389 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.699402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 754/1498] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:38.700496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.728409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.729384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.729665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.730492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.730666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.732640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.780141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48593fa-afba-4479-bd22-9181243a3778 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.784722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.785374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.785655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.785931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.786598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.788216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:38.799263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 755/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:38.801209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.801584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:38.830922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 745/1500] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:38 2022] POST /identity/v3/projects => generated 407 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:38.834640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:38.891345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecd66f-e61b-4ad0-89dc-ac518c433bbf admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.982734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7f98f-fccc-46c9-953b-0a6bc7dd0e28 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:38.987108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.987779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.988074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.988242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:38.988697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] PATH_INFO: `/v3/projects/05a0bb9574144fa4bd2ecc612d00b284/users/a5c6e38b8bb74c4ba4714df0613f6141/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.988867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:38.989082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:38.989363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:38.990292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a0bb9574144fa4bd2ecc612d00b284, user_id=a5c6e38b8bb74c4ba4714df0613f6141, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:38.992999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.015620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 746/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:38 2022] PUT /identity/v3/projects/05a0bb9574144fa4bd2ecc612d00b284/users/a5c6e38b8bb74c4ba4714df0613f6141/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:39.015620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.053617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.055101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b8670-0c17-4cca-8fea-14665c54e4fd service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.063896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:39.065556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 756/1502] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:39.067017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.067307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.067982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.080783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0d690-2960-45fc-8429-e5a87bffd191 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.087392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.087392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.087392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.087700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.088081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.090046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.097847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] MFA Rules not processed for user `a5c6e38b8bb74c4ba4714df0613f6141`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:39.099449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:39.169076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 757/1503] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:39.180692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.182750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:39.198462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 747/1504] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:39 2022] POST /identity/v3/users => generated 304 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:39.205067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.232069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a7159-bbbb-4335-86a1-d0179810949d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.237234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.237930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.238297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.238633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.239448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.241297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.256886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 748/1505] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:39.257886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.294462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1eaeef-8b98-4be0-8cda-fe6b1aff6309 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.298176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.298887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.299222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.299770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.309447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.310935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.322691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 758/1506] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:39.323413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.333854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a22b3-2cb6-48da-b9f6-6f3acc1de2ed service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.339736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.340272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.340547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.340816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.341761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.342216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbe2620-1692-484a-aa04-76d438a163d3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.343379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.349572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.351156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.351576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.351989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.354169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 759/1507] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:39.355549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:39.358070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:39.376288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9de18-e2f0-40f0-b978-4a89913bcdf0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:39.383144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:39.384337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:39.384617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:39.384886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] PATH_INFO: `/v3/projects/f7d2c9678ba8482095a7670ba0a972fa/users/a56af98f33714a0c9bb43b4dfbb3d76b/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:39.404460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d2c9678ba8482095a7670ba0a972fa, user_id=a56af98f33714a0c9bb43b4dfbb3d76b, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.407438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.448500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:39.449944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe14dbe-b497-4e67-b5e8-4097016d7b89 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:39.460014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 749/1508] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:40.459194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 760/1509] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:39 2022] PUT /identity/v3/projects/f7d2c9678ba8482095a7670ba0a972fa/users/a56af98f33714a0c9bb43b4dfbb3d76b/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 1101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:40.472414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:40.472889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:40.473037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:40.510149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] MFA Rules not processed for user `a56af98f33714a0c9bb43b4dfbb3d76b`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:40.511405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:40.578418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 750/1510] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:40 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:40.582895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:40.608595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:40.649599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc5615-2489-42f8-a9dd-62237b49eb0a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:40.654486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:40.655141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:40.655416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:40.655689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:40.657071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:40.659003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:40.742121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 761/1511] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:40 2022] POST /identity/v3/projects => generated 407 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:40.746970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:40.757406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53c834-fe40-4a07-b892-892f48db3dda None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:40.814939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:40.815476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:40.815808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:40.816075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:40.816693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:40.818599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:40.944995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 751/1512] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:40 2022] POST /identity/v3/users => generated 310 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:40.946058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:40.957103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b987-5c19-4f8c-bdbd-444b35a5c52e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:40.962200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:40.963144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:40.963428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:40.963698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:40.964341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:40.966096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:40.982186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:40.988519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 762/1513] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:40.992731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc25c21-d3b0-4d83-a7f9-c04e454dc706 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:41.001869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:41.002545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.007248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.007248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] PATH_INFO: `/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d/users/3aeb0d67d4a24a568ae0e384c269cba4/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.030184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd01dd54f7e42778c8ad32d0018e27d, user_id=3aeb0d67d4a24a568ae0e384c269cba4, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:41.033010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:41.191002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 752/1514] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:40 2022] PUT /identity/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d/users/3aeb0d67d4a24a568ae0e384c269cba4/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:41.196254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:41.223287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a96d4-fca5-4f4b-8de7-d4b2411232ca admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:41.359130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:41.359760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.360043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.360523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.360970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:41.362610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d5fb2-42c8-4cac-9252-bce25a19b8fb admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:41.374862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 763/1515] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:41.377121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:41.388273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29969589-e35c-4714-8758-cf61a682f645 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:41.400498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:41.400498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.400498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.400498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] PATH_INFO: `/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d/users/3aeb0d67d4a24a568ae0e384c269cba4/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.401606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd01dd54f7e42778c8ad32d0018e27d, user_id=3aeb0d67d4a24a568ae0e384c269cba4, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:41.402020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:41.405345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:41.421660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 753/1516] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:41 2022] PUT /identity/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d/users/3aeb0d67d4a24a568ae0e384c269cba4/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:41.441223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.441223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.441223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.467708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] MFA Rules not processed for user `3aeb0d67d4a24a568ae0e384c269cba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:41.469139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:41.548569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 764/1517] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:41 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:41.578386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:41.662251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a87b1f-e5f4-4a7e-9604-b55df3749579 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:41.666052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:41.666950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.667235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.667513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.680313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:41.681989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:41.692544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 754/1518] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:41 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:41.758249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:41.768137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06da3d-a48d-4cb5-bbdd-ebbb2f26568b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:41.771725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:41.772327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:41.772604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:41.772872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:41.774585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:41.835229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:41.836607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc50a1-6cc2-4443-8356-be9384011450 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:41.845354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 765/1519] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:42.023955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:42.316406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:42.331670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf1732-a2ea-42e8-8ca7-8efd605570f4 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:42.335594 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:42.336266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:42.336539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:42.336987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:42.402924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:42.404451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154933d3-a19d-42f8-9256-b18ec6a86358 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:42.415333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 755/1520] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:42.524764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:42.662944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:42.663846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:42.664266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:42.667031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 766/1521] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:18:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:18:42.683908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:42.684360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:42.684360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:42.725958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:42.729345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:42.821891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 756/1522] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:18:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:42.840877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:42.841467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:42.860819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0250ef3-1491-4f48-8eff-a105a0c80d31 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:42.866441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:42.867077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:42.867354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:42.867625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:42.876554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:42.878416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84967415-eeb3-4f31-a324-bdee61a63862 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:42.890496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 767/1523] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:43.536913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:44.548351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:45.001982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:45.574373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:45.575988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:45.640636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9543d0a-6696-481e-8aac-d30081385844 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:45.645373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:45.646112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:45.646394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:45.646670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] PATH_INFO: `/v3/users/41c9ef5de3d14dca94541d5786358789` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:45.647860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c9ef5de3d14dca94541d5786358789)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:45.649809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:45.739141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:45.915561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c9ef5de3d14dca94541d5786358789'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:45.922588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c9ef5de3d14dca94541d5786358789'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:45.923293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c9ef5de3d14dca94541d5786358789'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.064087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c9ef5de3d14dca94541d5786358789'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.064801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c9ef5de3d14dca94541d5786358789'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.084306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 757/1524] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:45 2022] DELETE /identity/v3/users/41c9ef5de3d14dca94541d5786358789 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:46.088953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:46.089412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:46.089502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:46.108401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:46.109584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:46.202089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 768/1525] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:46.217653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:46.217653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:46.299581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff140b-2c76-4785-a62e-e742fc3dbc04 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:46.304704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:46.305547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:46.305973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:46.306390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:46.320549 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:46.322123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:46.334871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 758/1526] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:46.758554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:46.761412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:46.762591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:46.775197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89bc06-8003-4018-85bb-d73dddb1acd1 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:46.780569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:46.781247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:46.781521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:46.781788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] PATH_INFO: `/v3/users/6206de16aa3145f7a03c605f48be3a96` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:46.782950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6206de16aa3145f7a03c605f48be3a96)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:46.784941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:46.848722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:46.851213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:46.866102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051d9d-b43a-46aa-9ab7-f77db03cf58e service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:46.870770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:46.871448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:46.871733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:46.872006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] PATH_INFO: `/v3/projects/c4f057b686054bcfb786edc7136c89a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:46.879529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f057b686054bcfb786edc7136c89a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:46.881498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:46.907994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:46.985709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6206de16aa3145f7a03c605f48be3a96'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.992799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6206de16aa3145f7a03c605f48be3a96'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.993490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6206de16aa3145f7a03c605f48be3a96'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:46.996291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f057b686054bcfb786edc7136c89a2', 'request_id': 'req-842ef00b-65d1-4c16-8edb-81a7c2c956ee'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.002484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f057b686054bcfb786edc7136c89a2', 'request_id': 'req-842ef00b-65d1-4c16-8edb-81a7c2c956ee'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.007991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f057b686054bcfb786edc7136c89a2', 'request_id': 'req-842ef00b-65d1-4c16-8edb-81a7c2c956ee'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.069982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6206de16aa3145f7a03c605f48be3a96'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.070595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f4642a94-961d-48a0-a272-728fbb4bc0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6206de16aa3145f7a03c605f48be3a96'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.089074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 769/1527] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:46 2022] DELETE /identity/v3/users/6206de16aa3145f7a03c605f48be3a96 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:47.093419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:47.093701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:47.093971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:47.112257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:47.113432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:47.126909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] The token cache is being invalidate because project c4f057b686054bcfb786edc7136c89a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:47.127206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.127944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.132550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 759/1528] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:46 2022] DELETE /identity/v3/projects/c4f057b686054bcfb786edc7136c89a2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:47.133788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:47.135951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:47.172076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 770/1529] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:47.183332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:47.184759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:47.200990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842ef00b-65d1-4c16-8edb-81a7c2c956ee admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:47.205627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:47.206265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:47.206616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:47.206920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] PATH_INFO: `/v3/users/9392335b781c4877b8bba258b4d17bb7` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:47.208049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9392335b781c4877b8bba258b4d17bb7)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:47.209976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:47.247039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e54735-c128-4593-84f2-d2ca98f51081 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:47.250919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:47.251566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:47.251846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:47.252118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:47.261893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:47.263363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:47.274968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 771/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:47.383789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9392335b781c4877b8bba258b4d17bb7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.391621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9392335b781c4877b8bba258b4d17bb7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.392359 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9392335b781c4877b8bba258b4d17bb7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.516331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9392335b781c4877b8bba258b4d17bb7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.517081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9392335b781c4877b8bba258b4d17bb7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:47.539316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 760/1531] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:47 2022] DELETE /identity/v3/users/9392335b781c4877b8bba258b4d17bb7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:47.547144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:47.785366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:47.911428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:47.918318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:47.931255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344badca-f060-4bfc-bf27-f1d7fbe04771 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.005604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.007551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.007830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.008163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] PATH_INFO: `/v3/projects/d14eecab70af421cbf506d397c9fca4c` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.009949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14eecab70af421cbf506d397c9fca4c)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.012005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.115458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.116942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:48.136597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0768e2b-7cca-46bb-b984-6520a10e631e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.145619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14eecab70af421cbf506d397c9fca4c', 'request_id': 'req-cbca848b-957e-4b34-ac96-cc884b0c0b22'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.150465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14eecab70af421cbf506d397c9fca4c', 'request_id': 'req-cbca848b-957e-4b34-ac96-cc884b0c0b22'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.156756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14eecab70af421cbf506d397c9fca4c', 'request_id': 'req-cbca848b-957e-4b34-ac96-cc884b0c0b22'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.183254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:48.193459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.194082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.194357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.194628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] PATH_INFO: `/v3/projects/037f21b1cc394b9bbb36555a12025a7a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.195743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=037f21b1cc394b9bbb36555a12025a7a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.197626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.207172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] The token cache is being invalidate because project d14eecab70af421cbf506d397c9fca4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:48.207524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.208161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.209822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 772/1532] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:47 2022] DELETE /identity/v3/projects/d14eecab70af421cbf506d397c9fca4c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:48.214233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.215882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:48.228099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca848b-957e-4b34-ac96-cc884b0c0b22 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.232988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.233654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.233940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.234216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] PATH_INFO: `/v3/users/3158691568524f2d89cb57c6aee6c372` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.235443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3158691568524f2d89cb57c6aee6c372)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.237508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.304454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '037f21b1cc394b9bbb36555a12025a7a', 'request_id': 'req-f6f2ca4c-64fa-4ddf-9ede-90e165946844'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.310742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '037f21b1cc394b9bbb36555a12025a7a', 'request_id': 'req-f6f2ca4c-64fa-4ddf-9ede-90e165946844'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.316419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '037f21b1cc394b9bbb36555a12025a7a', 'request_id': 'req-f6f2ca4c-64fa-4ddf-9ede-90e165946844'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.344816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:48.378397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] The token cache is being invalidate because project 037f21b1cc394b9bbb36555a12025a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:48.378669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.379340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6f2ca4c-64fa-4ddf-9ede-90e165946844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.380909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 761/1533] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:48 2022] DELETE /identity/v3/projects/037f21b1cc394b9bbb36555a12025a7a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:48.386129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536edd-bece-4f99-bc61-7150425378d1 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.386418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536edd-bece-4f99-bc61-7150425378d1 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.386739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29536edd-bece-4f99-bc61-7150425378d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.404959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3158691568524f2d89cb57c6aee6c372'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.408072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-29536edd-bece-4f99-bc61-7150425378d1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:48.409305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-29536edd-bece-4f99-bc61-7150425378d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:48.412416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3158691568524f2d89cb57c6aee6c372'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.413093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3158691568524f2d89cb57c6aee6c372'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.466556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 762/1534] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:48.474199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.474485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.474760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.481707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3158691568524f2d89cb57c6aee6c372'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.482337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3158691568524f2d89cb57c6aee6c372'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:48.494566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:48.495960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:48.502445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 773/1535] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:48 2022] DELETE /identity/v3/users/3158691568524f2d89cb57c6aee6c372 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:48.509873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 763/1536] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:48.511634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.577962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1186a-c7ae-401f-b72a-3b08f2ea623b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.584301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.584928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.585261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.585594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.588611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:48.588611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.588611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.600896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 774/1537] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:18:48.605023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.614881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b1546-afc2-4d0f-920c-804732271a4e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.619671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.620939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.621283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.621548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.622353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.624280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.692680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:48.730860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 764/1538] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:48 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:48.738510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.755388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab3cf4-759f-42cb-8b56-596926bdd490 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.830475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.831404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.831677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.831952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.832766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.838509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.931553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 775/1539] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:48 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:48.935150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.952645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cb6-d13f-44ca-9dc6-50d320d7e774 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.957203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.957810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.958091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.958364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.959034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.960632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:48.971146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 765/1540] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:48.975543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:48.985409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802481c5-976b-46d5-a7c2-10b72512bbb4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:48.989932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:48.990526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:48.990807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:48.991107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:48.991745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:48.993320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.005172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 776/1541] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:49.009900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.020144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc520971-9a79-4918-8bfb-c65dae9138d9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.025567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.026223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.027150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.027425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] PATH_INFO: `/v3/projects/0bcb9273129e4bdf9c513508485a4003/users/25d27df2e39b48cbba40a65eec1cbc60/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.042526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bcb9273129e4bdf9c513508485a4003, user_id=25d27df2e39b48cbba40a65eec1cbc60, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.045559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049fbbd-575b-4e96-9231-65df463c72f2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.061940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.063714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:49.079218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e43893-e89c-4ea4-a201-9c7f7c6477b5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.084520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.085156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.085450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.085764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] PATH_INFO: `/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.086924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca0b2b6c22a4075b69efcdf04c0409b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.089054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.126199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 766/1542] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:49 2022] PUT /identity/v3/projects/0bcb9273129e4bdf9c513508485a4003/users/25d27df2e39b48cbba40a65eec1cbc60/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:49.127813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.138423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.138757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.139099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.167288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] MFA Rules not processed for user `25d27df2e39b48cbba40a65eec1cbc60`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:49.168754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:49.218038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca0b2b6c22a4075b69efcdf04c0409b', 'request_id': 'req-f89686c5-f734-44e3-88b0-a524fca6430c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:49.225745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca0b2b6c22a4075b69efcdf04c0409b', 'request_id': 'req-f89686c5-f734-44e3-88b0-a524fca6430c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:49.232147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.235077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca0b2b6c22a4075b69efcdf04c0409b', 'request_id': 'req-f89686c5-f734-44e3-88b0-a524fca6430c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:49.238337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 767/1543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:49.243214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.302647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] The token cache is being invalidate because project aca0b2b6c22a4075b69efcdf04c0409b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:49.303001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:49.303644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f89686c5-f734-44e3-88b0-a524fca6430c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:49.305236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 777/1544] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:49 2022] DELETE /identity/v3/projects/aca0b2b6c22a4075b69efcdf04c0409b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:49.308464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31393e-382b-4811-a9b6-2f601870306c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.310355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98fe5c-51c1-4523-8df4-c52967bc7223 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.310640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98fe5c-51c1-4523-8df4-c52967bc7223 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.310945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98fe5c-51c1-4523-8df4-c52967bc7223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.330610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-af98fe5c-51c1-4523-8df4-c52967bc7223 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:49.331844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-af98fe5c-51c1-4523-8df4-c52967bc7223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:49.365701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.366358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.366661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.366962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.367592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.369728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.396145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 778/1545] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:49.403408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.403695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.403971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.425019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:49.426205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:49.434541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 779/1546] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:49.438435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.447222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c3c6-1bef-43dc-a330-2fb2b2bcebfa None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.452326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.452890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.453170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.453462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.454069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:49.454375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.456061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.467449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 780/1547] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:18:49.471185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 768/1548] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/projects => generated 385 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:49.471564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.474956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.476538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:49.481138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e7a3b-33a6-47fb-b493-5f026ecd95b6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.538947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.539552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.539831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.540102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.540200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68e7b0-fe6b-4cf6-a8f1-7ccb163cd523 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.540784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.542693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.545112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.545710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.545986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.546256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.546893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.548946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.579275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.645121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 781/1549] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:49.646134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.650812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 769/1550] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:49.652508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.662262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6589e-807a-4cc2-9f84-96c133f9d6c4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.712115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd480ad-334d-4938-b8c2-020a804a2ec5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.717513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.718120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.718526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.718831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.719466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.721663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.731485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.732251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.732533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.734266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.734382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.734647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.735705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.746490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 770/1551] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:49.750494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.761253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e3c28-a338-4c4a-b29c-40827039a653 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.766001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.766775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.767174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.767505 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] PATH_INFO: `/v3/projects/fa8c810265104feba637c52c2ed6b47b/users/4d785352f90046c8a9a3060873899cf6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.784064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8c810265104feba637c52c2ed6b47b, user_id=4d785352f90046c8a9a3060873899cf6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.790209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.814857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 782/1552] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:49 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:49.816700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.827919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbd29-2ee4-491c-9f4f-d60cee67dac2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.832641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.833234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.833511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.833783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.834428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.836280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.847762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 783/1553] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:49.851460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.861738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967a656-d826-411d-a1a1-1c0c82c02230 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.866299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.866937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.867214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.867485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.868105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.871267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:49.880874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 771/1554] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:49 2022] PUT /identity/v3/projects/fa8c810265104feba637c52c2ed6b47b/users/4d785352f90046c8a9a3060873899cf6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:49.882031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 784/1555] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:18:49.886846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.887091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:49.896551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34b8d2-9a22-49e1-acc2-ae85494a1e43 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.945532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.971150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f056a-4362-4cfd-b37f-b69932c72d30 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:49.974025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:49.981318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.981318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.981318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.981318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.981603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:49.982187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:49.982476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:49.982754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] PATH_INFO: `/v3/projects/0dae9dad8f63413488d0120ac7ed41da/users/5c7ad80b1d4b4f58a69e47a659bb339e/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:49.982894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:49.984731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.003096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 772/1556] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:50.004257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.006880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dae9dad8f63413488d0120ac7ed41da, user_id=5c7ad80b1d4b4f58a69e47a659bb339e, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.009929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f8559-49e9-4e9b-9ca6-7c0dc0c650bf admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.017381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c298ffa-686c-49cb-90b4-09bc68dd7057 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.022866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.023758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.027792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.027792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] PATH_INFO: `/v3/projects/fa8c810265104feba637c52c2ed6b47b/users/4d785352f90046c8a9a3060873899cf6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.027792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8c810265104feba637c52c2ed6b47b, user_id=4d785352f90046c8a9a3060873899cf6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.031954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b95f4-1271-404f-b5f6-fe71634b464a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.045489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 773/1557] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:50 2022] PUT /identity/v3/projects/fa8c810265104feba637c52c2ed6b47b/users/4d785352f90046c8a9a3060873899cf6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:50.057134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.057432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.058290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.101871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] MFA Rules not processed for user `4d785352f90046c8a9a3060873899cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:50.105062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:50.135408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 785/1558] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:49 2022] PUT /identity/v3/projects/0dae9dad8f63413488d0120ac7ed41da/users/5c7ad80b1d4b4f58a69e47a659bb339e/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:50.148387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.148666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.149408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.183720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] MFA Rules not processed for user `5c7ad80b1d4b4f58a69e47a659bb339e`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:50.185034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:50.220143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 774/1559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:50.231245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.234153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:50.261414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 786/1560] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:50.266128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.304864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112e688-971d-4fd7-916f-8db429d400ae None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.318989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:50.356853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f06cb-f598-45bf-8efb-2b3901bcf115 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.363037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.363307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.369168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.369168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.369168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.369168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.373883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.374567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.375002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.375343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.377223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:50.389831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:50.449218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.450678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.460217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 775/1561] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:50.491285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 787/1562] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:50 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:50.498957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.511297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d42a1-3635-4e44-9919-5f02c8c9f233 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.578355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.579004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.579283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.579550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.580233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.582138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.699696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 776/1563] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:50 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:50.703824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.714329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd9478-1b10-420d-bb32-114950ad3766 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.718768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.719388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.719688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.719954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.720587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.722251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.732883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 788/1564] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:50.736995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.747891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f8a5d-a5ba-4e2e-9860-d9ca82001a9c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.752794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.753455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.753737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.754011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] PATH_INFO: `/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8/users/d03d195b97f34142a6791b8c08066baf/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.770609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=288ae6a36a5a43e5afcb2d461dbe11b8, user_id=d03d195b97f34142a6791b8c08066baf, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.773417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.814044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:50.894783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 777/1565] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:50 2022] PUT /identity/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8/users/d03d195b97f34142a6791b8c08066baf/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:50.904260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:50.915233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408fc84-9438-4313-b73f-c7addb19101a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:50.984619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:50.985157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:50.985431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:50.985732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:50.986372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:50.988048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647f5cd-dfb3-4231-abc5-6ee7814c47ba admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:50.997920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 789/1566] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:51.002263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:51.012577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22730623-31cf-492e-b897-7d53c24032bd admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:51.017156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:51.017803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.018196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.018380 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] PATH_INFO: `/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8/users/d03d195b97f34142a6791b8c08066baf/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.020559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288ae6a36a5a43e5afcb2d461dbe11b8, user_id=d03d195b97f34142a6791b8c08066baf, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:51.033154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:51.050875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 778/1567] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:51 2022] PUT /identity/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8/users/d03d195b97f34142a6791b8c08066baf/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:51.061110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.061695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.061989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.091360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] MFA Rules not processed for user `d03d195b97f34142a6791b8c08066baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:51.092621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:51.149807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:51.152891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:51.155029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:51.173200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 790/1568] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:51.215263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:51.225519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefc255-4517-444d-9c9f-b5cbe8532d74 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:51.235221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:51.235833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.236566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.236918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.239152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:51.265600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:51.275602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf61232-d089-4558-9cdd-8d0e2caa62f7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:51.280232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:51.280976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.281257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.281711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.292665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:51.294377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:51.308710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 791/1569] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:51.311144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:51.312931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ebb2f-c45f-4213-bdcb-504cd5619fff service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:51.324357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 779/1570] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:51.348667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:51.469009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83e822-8900-442f-80d8-358a98b11f50 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:51.473686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:51.474294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.474571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.474863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] PATH_INFO: `/v3/users/5c7ad80b1d4b4f58a69e47a659bb339e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.476024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7ad80b1d4b4f58a69e47a659bb339e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:51.478572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:51.481418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:51.721361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7ad80b1d4b4f58a69e47a659bb339e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:51.729120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7ad80b1d4b4f58a69e47a659bb339e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:51.729824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7ad80b1d4b4f58a69e47a659bb339e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:51.849067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7ad80b1d4b4f58a69e47a659bb339e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:51.849632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7ad80b1d4b4f58a69e47a659bb339e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:51.866704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:51.872710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 792/1571] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:51 2022] DELETE /identity/v3/users/5c7ad80b1d4b4f58a69e47a659bb339e => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:51.877182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:51.877465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:51.877787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:51.899449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:51.900748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:51.962662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 780/1572] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:51.972374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:51.973771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:52.031128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39b041-444c-4d4b-a22b-9f593cddc09f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:52.034765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:52.035760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:52.035760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:52.035996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:52.047969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:52.049432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:52.061293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 793/1573] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:52.794727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:52.796837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:52.827285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505d895-7f56-43b2-b57e-40e2fba4c15b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:52.833795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a427b-c008-417a-807c-73465399d75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:52.834698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a427b-c008-417a-807c-73465399d75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:52.835083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a427b-c008-417a-807c-73465399d75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:52.836072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a427b-c008-417a-807c-73465399d75e admin admin] PATH_INFO: `/v3/projects/0dae9dad8f63413488d0120ac7ed41da` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:52.837243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a427b-c008-417a-807c-73465399d75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dae9dad8f63413488d0120ac7ed41da)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:52.840583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a427b-c008-417a-807c-73465399d75e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:52.971247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dae9dad8f63413488d0120ac7ed41da', 'request_id': 'req-437a427b-c008-417a-807c-73465399d75e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:52.978338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dae9dad8f63413488d0120ac7ed41da', 'request_id': 'req-437a427b-c008-417a-807c-73465399d75e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:52.985562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dae9dad8f63413488d0120ac7ed41da', 'request_id': 'req-437a427b-c008-417a-807c-73465399d75e'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:52.988685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:53.047307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] The token cache is being invalidate because project 0dae9dad8f63413488d0120ac7ed41da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:53.047528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.048161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.049683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 781/1574] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:52 2022] DELETE /identity/v3/projects/0dae9dad8f63413488d0120ac7ed41da => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:53.053720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:53.065063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11576f0-17cc-4d12-8932-57bb44d0ce1a service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:53.120660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:53.121921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:53.122204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:53.122540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] PATH_INFO: `/v3/users/d03d195b97f34142a6791b8c08066baf` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:53.123606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03d195b97f34142a6791b8c08066baf)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:53.125646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:53.326883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03d195b97f34142a6791b8c08066baf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.349535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03d195b97f34142a6791b8c08066baf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.350331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03d195b97f34142a6791b8c08066baf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.460330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03d195b97f34142a6791b8c08066baf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.461130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-41fd8627-5465-4e2c-b9b2-84016d853d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03d195b97f34142a6791b8c08066baf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:53.480863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 794/1575] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:53 2022] DELETE /identity/v3/users/d03d195b97f34142a6791b8c08066baf => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:54.103390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:54.106836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:54.130317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a427b-c008-417a-807c-73465399d75e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:54.223511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:54.224148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.224424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.224721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] PATH_INFO: `/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.225875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=288ae6a36a5a43e5afcb2d461dbe11b8)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:54.227784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:54.353838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288ae6a36a5a43e5afcb2d461dbe11b8', 'request_id': 'req-5985d23f-bcbe-420c-a4e2-aeec2aa22046'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.360712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288ae6a36a5a43e5afcb2d461dbe11b8', 'request_id': 'req-5985d23f-bcbe-420c-a4e2-aeec2aa22046'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.366075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288ae6a36a5a43e5afcb2d461dbe11b8', 'request_id': 'req-5985d23f-bcbe-420c-a4e2-aeec2aa22046'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.487281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] The token cache is being invalidate because project 288ae6a36a5a43e5afcb2d461dbe11b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:54.487670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.488492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.490083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 782/1576] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:54 2022] DELETE /identity/v3/projects/288ae6a36a5a43e5afcb2d461dbe11b8 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:54.496876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb461-fc88-4f79-8c79-a8c5ca582de5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.497345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb461-fc88-4f79-8c79-a8c5ca582de5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.497420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb461-fc88-4f79-8c79-a8c5ca582de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.518218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ba7eb461-fc88-4f79-8c79-a8c5ca582de5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:54.519195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:54.519458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ba7eb461-fc88-4f79-8c79-a8c5ca582de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:54.520863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:54.534334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:54.587734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 795/1577] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:54.596243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.596528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.596800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.597118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985d23f-bcbe-420c-a4e2-aeec2aa22046 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:54.602789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:54.608842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.608842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.608842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] PATH_INFO: `/v3/users/a56af98f33714a0c9bb43b4dfbb3d76b` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.609971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a56af98f33714a0c9bb43b4dfbb3d76b)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:54.612098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:54.618555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:54.619641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:54.630149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 796/1578] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:54.634809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:54.646302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c60d9d-c57e-47e6-bc5b-1c0009226085 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:54.651232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:54.658665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:54.667257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 797/1579] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:18:54.672343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:54.681721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbca32d-5cee-40ce-a55d-970a3660fec0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:54.687382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:54.687382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.687382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.687758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.688160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:54.690095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:54.714070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:54.801051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 798/1580] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:54 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:54.802070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:54.863888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a56af98f33714a0c9bb43b4dfbb3d76b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.869032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a56af98f33714a0c9bb43b4dfbb3d76b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.869890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a56af98f33714a0c9bb43b4dfbb3d76b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.878355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5ef73-2b56-45a7-8348-726e16cc4b7a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:54.883689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:54.884308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:54.884589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:54.884863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:54.885498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:54.887550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:54.995942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a56af98f33714a0c9bb43b4dfbb3d76b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:54.996844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1dbccc8f-46bd-4ef5-a19d-c0e9278608ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a56af98f33714a0c9bb43b4dfbb3d76b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:55.005210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 799/1581] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:54 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:55.007464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.009353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:55.024323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 783/1582] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:54 2022] DELETE /identity/v3/users/a56af98f33714a0c9bb43b4dfbb3d76b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:55.029155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.030560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.030848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.036571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55fc45-5937-455c-9bea-ce5fd16b75f0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.055539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:55.056918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:55.111549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.112170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.112513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.112792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.113484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.115458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.127887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 784/1583] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:55.128565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 800/1584] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:55.129885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.140112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539a14d-2197-4a13-9f97-215ef0d1f4ff None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.142787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.144571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:55.147197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.147814 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.148151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.148430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.149077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.153307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.165766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 785/1585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:55.166929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.181854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab90b2-32dc-4853-a1a5-2fe602db9efe admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.184560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.185294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.185574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.186017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] PATH_INFO: `/v3/projects/2b140b93fe7044649980c0c0c6c631b7/users/a9e25b24eb70482b9309d7d8da1f5701/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.204760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b140b93fe7044649980c0c0c6c631b7, user_id=a9e25b24eb70482b9309d7d8da1f5701, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.210181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6013c9-d1c4-431b-86ef-b0709b73b4fd admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.210495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.211806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.212625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.212744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.212955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.223149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.224588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee75b9-8a9d-4a07-b46f-ef4cfe65ab11 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.235587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 801/1586] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:55.289089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 786/1587] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:55 2022] PUT /identity/v3/projects/2b140b93fe7044649980c0c0c6c631b7/users/a9e25b24eb70482b9309d7d8da1f5701/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:55.302189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.302673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.303203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.335864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] MFA Rules not processed for user `a9e25b24eb70482b9309d7d8da1f5701`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:55.338411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:55.433327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 802/1588] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:55.436356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:55.438170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.500783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252b7d-c057-43ea-a08b-48af6995b29d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.505730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.506369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.506649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.506967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.507645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.509646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.617103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:55.625442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 787/1589] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:55 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:55.629401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.641215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1db2c-2fd3-4767-a481-6deed7bdff49 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.698591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.699325 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.700216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.700560 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.701274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.703308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.823082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 803/1590] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:55 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:55.834298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.848031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d8a60-7742-4d24-b6d5-a8a199b88a19 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.853357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.854019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.863545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.863545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.863545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.863545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:55.868513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 788/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:55.873280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:55.886169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0a0a13-b331-4b7c-9989-5009aa596763 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:55.888819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:55.889492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:55.889776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:55.890057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] PATH_INFO: `/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63/users/c9fa659913ad4bf08674447e922eec3e/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:55.905983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3c96f39ca4f67a8099a4ce849fa63, user_id=c9fa659913ad4bf08674447e922eec3e, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:55.908735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.010702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 804/1592] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:55 2022] PUT /identity/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63/users/c9fa659913ad4bf08674447e922eec3e/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:56.012872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:56.026425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe1041-43ed-4622-ba3a-b22ab6e5a72a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:56.107623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:56.108390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.108764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.109138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.109887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:56.111744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.133737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 789/1593] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:56.138202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:56.143849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:56.149431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba406dbf-1bf2-49b8-ab5f-9c8c7b576885 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:56.155637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:56.156417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.156737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.157148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] PATH_INFO: `/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63/users/c9fa659913ad4bf08674447e922eec3e/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.159568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de3c96f39ca4f67a8099a4ce849fa63, user_id=c9fa659913ad4bf08674447e922eec3e, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:56.162849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e122b-d097-411b-958c-05c46fd3db3b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.172348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:56.174516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:56.179426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 805/1594] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:56 2022] PUT /identity/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63/users/c9fa659913ad4bf08674447e922eec3e/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:56.197037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da05962-e4f6-4c85-b846-ea231d347d80 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:56.211258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.212978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.215020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.260980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] MFA Rules not processed for user `c9fa659913ad4bf08674447e922eec3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:56.261813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:56.262429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:56.263204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.264448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.264808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] PATH_INFO: `/v3/projects/f7d2c9678ba8482095a7670ba0a972fa` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.265968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d2c9678ba8482095a7670ba0a972fa)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:56.268057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.342899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 806/1595] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:56.353666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:56.355054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:56.404295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d2c9678ba8482095a7670ba0a972fa', 'request_id': 'req-d790be19-f5ca-4d10-b55d-1ef938d87b81'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:56.412184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d2c9678ba8482095a7670ba0a972fa', 'request_id': 'req-d790be19-f5ca-4d10-b55d-1ef938d87b81'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:56.419843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d2c9678ba8482095a7670ba0a972fa', 'request_id': 'req-d790be19-f5ca-4d10-b55d-1ef938d87b81'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:56.420459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06901e0-ceb1-4519-83a1-459e9ffbea92 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:56.484965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:56.485624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.485905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.486179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.508143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] The token cache is being invalidate because project f7d2c9678ba8482095a7670ba0a972fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:56.508436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:56.509121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:56.510747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 790/1596] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:56 2022] DELETE /identity/v3/projects/f7d2c9678ba8482095a7670ba0a972fa => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:56.515202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:56.516761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:56.575009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:56.575965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:56.577135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.589220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 807/1597] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:56.690222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790be19-f5ca-4d10-b55d-1ef938d87b81 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:56.700163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:56.700824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:56.701107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:56.701381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] PATH_INFO: `/v3/users/3aeb0d67d4a24a568ae0e384c269cba4` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:56.703661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aeb0d67d4a24a568ae0e384c269cba4)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:56.705710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:56.901133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:57.003135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aeb0d67d4a24a568ae0e384c269cba4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:57.010928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aeb0d67d4a24a568ae0e384c269cba4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:57.011808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aeb0d67d4a24a568ae0e384c269cba4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:57.127484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aeb0d67d4a24a568ae0e384c269cba4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:57.128120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-4dc66b6c-f692-45ee-9311-cdcd8eddf314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aeb0d67d4a24a568ae0e384c269cba4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:57.147184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 791/1598] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:56 2022] DELETE /identity/v3/users/3aeb0d67d4a24a568ae0e384c269cba4 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:18:57.716370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:57.737402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007cab5-b0e3-43ef-9a9e-374978ce6949 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:57.874930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:57.875579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:57.875861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:57.876134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] PATH_INFO: `/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:57.877454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd01dd54f7e42778c8ad32d0018e27d)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:57.879407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.031312 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd01dd54f7e42778c8ad32d0018e27d', 'request_id': 'req-ffe001be-7371-42dc-ae97-f391397e738b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.039694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd01dd54f7e42778c8ad32d0018e27d', 'request_id': 'req-ffe001be-7371-42dc-ae97-f391397e738b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.046009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd01dd54f7e42778c8ad32d0018e27d', 'request_id': 'req-ffe001be-7371-42dc-ae97-f391397e738b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.125005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] The token cache is being invalidate because project ddd01dd54f7e42778c8ad32d0018e27d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:18:58.125377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.126291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.128048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 808/1599] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:57 2022] DELETE /identity/v3/projects/ddd01dd54f7e42778c8ad32d0018e27d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:58.132809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916e09-9d74-4a1f-9058-709fa365ede4 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.133278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916e09-9d74-4a1f-9058-709fa365ede4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.134179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916e09-9d74-4a1f-9058-709fa365ede4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.143387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.148378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.161762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8f916e09-9d74-4a1f-9058-709fa365ede4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:58.163548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8f916e09-9d74-4a1f-9058-709fa365ede4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.237340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe001be-7371-42dc-ae97-f391397e738b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.251031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.251660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.251939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.252210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] PATH_INFO: `/v3/users/a9e25b24eb70482b9309d7d8da1f5701` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.255132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e25b24eb70482b9309d7d8da1f5701)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.256794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.275110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:58.303438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 792/1600] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:58.311532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.312075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.312534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.363925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:58.369999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.394455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 793/1601] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:18:58.402045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.413159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56651a7e-6aa5-4696-9503-7bd14ee2ff4d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.418977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.424582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.442980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 794/1602] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:18:58.445958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.457089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d6305-e4fc-4e20-88cb-ae364fe4689e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.468409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.469114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.469473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.469816 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.470589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.472881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.479966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:58.533552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e25b24eb70482b9309d7d8da1f5701'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.541265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e25b24eb70482b9309d7d8da1f5701'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.542140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e25b24eb70482b9309d7d8da1f5701'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.603597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 795/1603] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:58 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:18:58.605512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.607690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.621108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e25b24eb70482b9309d7d8da1f5701'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.622270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e25b24eb70482b9309d7d8da1f5701'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.647782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 809/1604] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:58 2022] DELETE /identity/v3/users/a9e25b24eb70482b9309d7d8da1f5701 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:58.647782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.649505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.698148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8d7b7-bc30-4bfa-8bd8-f59bb6f9ead8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.729573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743225a-e6c9-4868-9cd6-fed5ed198bda admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.736180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.737166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.737667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.746995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.746995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.746995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac89e34-25aa-4c74-ae6a-829d91d5c473 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.777398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.777398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.777398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.777398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] PATH_INFO: `/v3/users/9ae331bc8d2449619c782347a2e65c8b` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.779406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae331bc8d2449619c782347a2e65c8b)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.781305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.851423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 810/1605] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:58 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:58.852722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.853159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.853512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.883209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:58.884540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:58.917260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 811/1606] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:58.917260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.930487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339a9ec-4daa-45ef-957d-584187b4cb8d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:58.936634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:58.942850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:58.942850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:58.942850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:58.942850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:58.942850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:58.954557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae331bc8d2449619c782347a2e65c8b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.971316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae331bc8d2449619c782347a2e65c8b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.972090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae331bc8d2449619c782347a2e65c8b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:58.977887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 812/1607] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:58.980322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:58.984306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.044827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6becae-9950-410c-afc4-6342333b4f2b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.048800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.049408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.049714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.049993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.051932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.072510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae331bc8d2449619c782347a2e65c8b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:59.073245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae331bc8d2449619c782347a2e65c8b'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:18:59.092364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 796/1608] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:18:58 2022] DELETE /identity/v3/users/9ae331bc8d2449619c782347a2e65c8b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:59.093335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.094683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.128449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.129913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.141215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 813/1609] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:59.152621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.162051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dddc4-0e16-4f82-9729-606a6eee4592 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.165993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c5554a-227b-407e-abc7-137b9a7eb783 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.168108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.168760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.169288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.169568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.172434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.236980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.237590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.237870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.238279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.238955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.240582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70ce6c-524e-43b8-b258-2bdded3ce50d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.251347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 814/1610] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:59.252672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.253166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.253786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.255992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.255992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.266611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 797/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:18:59.267300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.268756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.275226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:59.277004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.285339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f61b56-3a8b-4007-8e7f-587a4cf024e3 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.290769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 815/1612] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:59.293138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.301740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.302038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.302330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] PATH_INFO: `/v3/projects/3b2390694559449eb2326571db326999/users/83a92dc9e3cf41e6b4482e5a6b58b129/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.302433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.305194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.319739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2390694559449eb2326571db326999, user_id=83a92dc9e3cf41e6b4482e5a6b58b129, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.322458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.324465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698b9df-c1eb-4180-afce-b8d41a9c4a41 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.329975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.330692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.331063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.331418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.340903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.342306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf248079-15f3-43bb-b52c-ea6100e525b6 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.351708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 816/1613] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:18:59.397324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 798/1614] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:18:59 2022] PUT /identity/v3/projects/3b2390694559449eb2326571db326999/users/83a92dc9e3cf41e6b4482e5a6b58b129/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:18:59.409180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.409555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.409912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.438578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] MFA Rules not processed for user `83a92dc9e3cf41e6b4482e5a6b58b129`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:18:59.439821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.509608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 817/1615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:18:59.510882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.578558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8998c0e-fcd1-482f-b577-42c587af38cc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.583970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.584646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.585058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.585339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.586382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.587955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.686672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 799/1616] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:18:59 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:59.690531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.705838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256a64d-0913-4ead-a310-6dfd59d66051 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.760917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:18:59.781376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.781998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.782280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.782554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.783538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.785456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.933111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 818/1617] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:18:59 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:18:59.934410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.936045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.936759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:18:59.948365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd23019-57d5-4ac8-8fe8-1e25f16a6083 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.952512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2ede8-3009-4cdc-8ee9-e2c481efe8b5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:18:59.957478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.957478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.958151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.958151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.958986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.960694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.972353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:18:59.980784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 819/1618] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:18:59.985166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:18:59.986323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:18:59.986482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] PATH_INFO: `/v3/projects/2b140b93fe7044649980c0c0c6c631b7` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:18:59.987592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:18:59.989293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b140b93fe7044649980c0c0c6c631b7)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:18:59.991670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:18:59.996884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:00.049315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69504a59-d37e-4249-89e9-fa4b3e6a979e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:00.056676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:00.057688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:00.058185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:00.058662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] PATH_INFO: `/v3/projects/9d2df5f568ea42fcbe400803c83ee1fe` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:00.061287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2df5f568ea42fcbe400803c83ee1fe)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:00.064458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:00.171894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b140b93fe7044649980c0c0c6c631b7', 'request_id': 'req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.180019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b140b93fe7044649980c0c0c6c631b7', 'request_id': 'req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.188536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b140b93fe7044649980c0c0c6c631b7', 'request_id': 'req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.248998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2df5f568ea42fcbe400803c83ee1fe', 'request_id': 'req-c55ac002-176e-430c-be4f-8adafe789615'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.255484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2df5f568ea42fcbe400803c83ee1fe', 'request_id': 'req-c55ac002-176e-430c-be4f-8adafe789615'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.262128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2df5f568ea42fcbe400803c83ee1fe', 'request_id': 'req-c55ac002-176e-430c-be4f-8adafe789615'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.268681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:00.302090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] The token cache is being invalidate because project 2b140b93fe7044649980c0c0c6c631b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:00.302388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.308860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.311201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 800/1619] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:59 2022] DELETE /identity/v3/projects/2b140b93fe7044649980c0c0c6c631b7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:00.312645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:00.314232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:00.389468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecab61e-fd50-4e5f-b3bc-a455eb2b3237 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:00.424442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] The token cache is being invalidate because project 9d2df5f568ea42fcbe400803c83ee1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:00.424665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.425414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.427660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 820/1620] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:18:59 2022] DELETE /identity/v3/projects/9d2df5f568ea42fcbe400803c83ee1fe => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:00.427884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:00.431829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:00.458735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:00.459981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:00.459981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:00.460405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] PATH_INFO: `/v3/projects/547bac1868064923b0e9096834a7f0e2/users/9754582e3f8e4f9ca0c774a9921ea4de/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:00.478391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547bac1868064923b0e9096834a7f0e2, user_id=9754582e3f8e4f9ca0c774a9921ea4de, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:00.481121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:00.504662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac002-176e-430c-be4f-8adafe789615 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:00.509295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:00.509921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:00.510204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:00.510480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] PATH_INFO: `/v3/users/c9fa659913ad4bf08674447e922eec3e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:00.511676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9fa659913ad4bf08674447e922eec3e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:00.514161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:00.559185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 801/1621] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:00 2022] PUT /identity/v3/projects/547bac1868064923b0e9096834a7f0e2/users/9754582e3f8e4f9ca0c774a9921ea4de/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:00.560554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:00.562096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:00.627020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a8989-9f15-493f-b6aa-f7f03519ffd5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:00.631691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:00.632322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:00.632622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:00.632903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] PATH_INFO: `/v3/users/3c828fd5ff754beaaa34617c669d11ae` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:00.634088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c828fd5ff754beaaa34617c669d11ae)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:00.636061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:00.734254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9fa659913ad4bf08674447e922eec3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.744731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9fa659913ad4bf08674447e922eec3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.744731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9fa659913ad4bf08674447e922eec3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.856449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9fa659913ad4bf08674447e922eec3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.857274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9fa659913ad4bf08674447e922eec3e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.875498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c828fd5ff754beaaa34617c669d11ae'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.877237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 821/1622] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:00 2022] DELETE /identity/v3/users/c9fa659913ad4bf08674447e922eec3e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:00.878275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:00.880053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:00.887237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c828fd5ff754beaaa34617c669d11ae'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.888620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c828fd5ff754beaaa34617c669d11ae'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.953016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c828fd5ff754beaaa34617c669d11ae'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.953458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742662-9eb5-4dd3-abb6-c226d3d74895 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:00.953983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b87e303d-14b2-4e86-9fb1-66a31d8bac42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c828fd5ff754beaaa34617c669d11ae'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:00.959249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:00.959926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:00.960208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:00.960485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:00.961138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:00.962999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:00.975918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 822/1623] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:00.982609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 802/1624] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:00 2022] DELETE /identity/v3/users/3c828fd5ff754beaaa34617c669d11ae => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:00.988327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:00.990702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:01.007770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b51fdea-a0be-4402-b278-8036e1c6f2d1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:01.096440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:01.110945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.110945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.110945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] PATH_INFO: `/v3/projects/547bac1868064923b0e9096834a7f0e2/users/9754582e3f8e4f9ca0c774a9921ea4de/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.110945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=547bac1868064923b0e9096834a7f0e2, user_id=9754582e3f8e4f9ca0c774a9921ea4de, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:01.110945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:01.118434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 823/1625] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:00 2022] PUT /identity/v3/projects/547bac1868064923b0e9096834a7f0e2/users/9754582e3f8e4f9ca0c774a9921ea4de/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:01.139644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.139932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.140213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.173305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] MFA Rules not processed for user `9754582e3f8e4f9ca0c774a9921ea4de`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:01.175334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:01.260997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 803/1626] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:01.269839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:01.272892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:01.358664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:01.365143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8e2c1-e245-4420-8f85-30fedbd19330 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:01.369826 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:01.370623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.370909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.371259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.390431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:01.390431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:01.399941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 824/1627] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:01.480174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:01.480622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:01.581388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:01.582796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:01.593221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f84b6-b60c-4f48-9ac9-4c31e47bdfe6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:01.612348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74989db8-edce-4e79-a73c-bd83eca1d07d service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:01.619677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:01.620666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.620971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.621257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] PATH_INFO: `/v3/projects/22f753cd06384ef4b71a355e648de6f3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.622590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f753cd06384ef4b71a355e648de6f3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:01.624643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:01.634350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:01.635874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.636748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.637170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] PATH_INFO: `/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.640024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de3c96f39ca4f67a8099a4ce849fa63)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:01.645928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:01.792923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:01.823247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de3c96f39ca4f67a8099a4ce849fa63', 'request_id': 'req-ce7ca10e-e677-4b69-976e-3b0ea750712a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.828719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f753cd06384ef4b71a355e648de6f3', 'request_id': 'req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.838416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de3c96f39ca4f67a8099a4ce849fa63', 'request_id': 'req-ce7ca10e-e677-4b69-976e-3b0ea750712a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.838648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f753cd06384ef4b71a355e648de6f3', 'request_id': 'req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.846779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:01.848188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de3c96f39ca4f67a8099a4ce849fa63', 'request_id': 'req-ce7ca10e-e677-4b69-976e-3b0ea750712a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.850151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f753cd06384ef4b71a355e648de6f3', 'request_id': 'req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.945648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] The token cache is being invalidate because project 0de3c96f39ca4f67a8099a4ce849fa63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:01.947374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.949489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ce7ca10e-e677-4b69-976e-3b0ea750712a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:01.949784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 804/1628] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:01 2022] DELETE /identity/v3/projects/0de3c96f39ca4f67a8099a4ce849fa63 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:01.954867 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604f11-7393-4ef7-9c89-1a3e98d8499c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:01.955170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604f11-7393-4ef7-9c89-1a3e98d8499c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:01.955432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604f11-7393-4ef7-9c89-1a3e98d8499c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:01.975600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-10604f11-7393-4ef7-9c89-1a3e98d8499c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:01.976807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-10604f11-7393-4ef7-9c89-1a3e98d8499c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.027565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] The token cache is being invalidate because project 22f753cd06384ef4b71a355e648de6f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:02.028112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.029450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.031289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 825/1629] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:01 2022] DELETE /identity/v3/projects/22f753cd06384ef4b71a355e648de6f3 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:02.035129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.036632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.038771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 805/1630] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:02.046875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.052504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.052504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.063324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312787-9e8e-4c9d-8e45-8bd1b55bb4b5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.069533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.069822 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:02.070266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.070552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.070764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.070863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] PATH_INFO: `/v3/users/a5c6e38b8bb74c4ba4714df0613f6141` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.074980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c6e38b8bb74c4ba4714df0613f6141)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.075302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.080418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 806/1631] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:02.081246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.083397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.095180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf0cde-c357-4aa4-add0-bd0cd56f8385 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.099829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.100525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.100764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.101043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] PATH_INFO: `/v3/users/83a92dc9e3cf41e6b4482e5a6b58b129` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.102152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a92dc9e3cf41e6b4482e5a6b58b129)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.104131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.294070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c6e38b8bb74c4ba4714df0613f6141'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.307229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c6e38b8bb74c4ba4714df0613f6141'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.308031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c6e38b8bb74c4ba4714df0613f6141'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.323780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a92dc9e3cf41e6b4482e5a6b58b129'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.334432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a92dc9e3cf41e6b4482e5a6b58b129'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.335789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a92dc9e3cf41e6b4482e5a6b58b129'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.372096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c6e38b8bb74c4ba4714df0613f6141'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.373923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c6e38b8bb74c4ba4714df0613f6141'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.397748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 826/1632] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:02 2022] DELETE /identity/v3/users/a5c6e38b8bb74c4ba4714df0613f6141 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:02.399008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.401894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.444890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a92dc9e3cf41e6b4482e5a6b58b129'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.445749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-09796a05-7f68-4374-a959-24c090e910a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a92dc9e3cf41e6b4482e5a6b58b129'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:02.470434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 807/1633] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:02 2022] DELETE /identity/v3/users/83a92dc9e3cf41e6b4482e5a6b58b129 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:02.474959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.475311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.475597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.489014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ace2e-6cb4-49e1-93be-98832d6f253f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.504327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:02.506079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.559484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.560643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.560923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.561195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.561985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:02.562293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.564354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.578854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 827/1634] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:02.580256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.582073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.582807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 808/1635] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:02.594372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e64b4-3c4d-410e-a820-1c6d3e87d601 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.595614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.597444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:02.599872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.601642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.601642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.601642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.601903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.608036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.676705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e104d-9b21-4ebf-9c03-5121a864376d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.680499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.681125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.681426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.681709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.694540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.696994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.701973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 828/1636] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:02 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:02.703339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.731732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 809/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:02.791598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36798bab-ddfc-4cb2-b337-2f32e848a307 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.799689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.800464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.800822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.801188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.802508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.804643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.826349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:02.946847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 829/1638] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:02 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:02.948485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:02.959332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e5e-c721-499a-b292-49190c74bd5a service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:02.965535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:02.966196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:02.967025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:02.967328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:02.972189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:02.974360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:02.988614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 810/1639] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:02.993629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.027642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd62325-d77a-4b1b-9f72-d8290e5d9dfd admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:03.051923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:03.052587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.052861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.053229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.053855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:03.055545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:03.066213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 830/1640] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:03.070551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.082789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74427d7-e8a7-4b65-ae24-8374b36c93b0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:03.087863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:03.090464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.090464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.090464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] PATH_INFO: `/v3/projects/fcc46bf86cf044be81715a4c51d58c62/users/06cbac6e08094f2a8667f74799352868/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.107754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc46bf86cf044be81715a4c51d58c62, user_id=06cbac6e08094f2a8667f74799352868, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:03.110734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2280c3-bbe1-4a73-80c6-ea49cca13a59 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:03.137876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.151686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.167754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ac416-31b0-4f26-a479-df709413719e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:03.173660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:03.174320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.174589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.174888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] PATH_INFO: `/v3/projects/05a0bb9574144fa4bd2ecc612d00b284` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.176267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a0bb9574144fa4bd2ecc612d00b284)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:03.178081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:03.215347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 811/1641] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:03 2022] PUT /identity/v3/projects/fcc46bf86cf044be81715a4c51d58c62/users/06cbac6e08094f2a8667f74799352868/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:03.226927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.227615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.228082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.256832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] MFA Rules not processed for user `06cbac6e08094f2a8667f74799352868`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:03.258052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.364119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a0bb9574144fa4bd2ecc612d00b284', 'request_id': 'req-19c9df38-7547-424f-8f30-bdf48311e3b4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.383936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a0bb9574144fa4bd2ecc612d00b284', 'request_id': 'req-19c9df38-7547-424f-8f30-bdf48311e3b4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.394598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a0bb9574144fa4bd2ecc612d00b284', 'request_id': 'req-19c9df38-7547-424f-8f30-bdf48311e3b4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.409884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 812/1642] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:03.410974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.412717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.491778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941f3a1-8fb2-4c33-ab8b-94904fbc3306 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:03.519900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] The token cache is being invalidate because project 05a0bb9574144fa4bd2ecc612d00b284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:03.520214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.523326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.526685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 831/1643] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:03 2022] DELETE /identity/v3/projects/05a0bb9574144fa4bd2ecc612d00b284 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:03.528494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.535077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.572434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:03.572926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.573289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.584305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] PATH_INFO: `/v3/projects/3b2390694559449eb2326571db326999` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.584514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2390694559449eb2326571db326999)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:03.586355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:03.641097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9df38-7547-424f-8f30-bdf48311e3b4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:03.646250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:03.646906 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.647190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.647465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.651114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.698968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:03.720167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:03.721941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da9026-cb68-429a-889d-33bb4608bb42 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:03.772141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2390694559449eb2326571db326999', 'request_id': 'req-3b5b8431-3eec-4580-807f-ec7f33d8187c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.780129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2390694559449eb2326571db326999', 'request_id': 'req-3b5b8431-3eec-4580-807f-ec7f33d8187c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.786040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 832/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:03.787999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba66f4-9da8-418d-ac34-76887719c073 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.788119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba66f4-9da8-418d-ac34-76887719c073 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.788387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba66f4-9da8-418d-ac34-76887719c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:03.793497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2390694559449eb2326571db326999', 'request_id': 'req-3b5b8431-3eec-4580-807f-ec7f33d8187c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.831672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-38ba66f4-9da8-418d-ac34-76887719c073 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:03.837459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-38ba66f4-9da8-418d-ac34-76887719c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.890903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] The token cache is being invalidate because project 3b2390694559449eb2326571db326999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:03.891523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.892661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:03.895607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 813/1645] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:03 2022] DELETE /identity/v3/projects/3b2390694559449eb2326571db326999 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:03.899608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:03.901286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:03.957458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 833/1646] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:03.965828 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:03.966777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:03.966777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.009162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:04.010988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:04.021636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 834/1647] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:04.033224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.036266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8431-3eec-4580-807f-ec7f33d8187c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.044329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.045447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.045732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.046414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] PATH_INFO: `/v3/users/9754582e3f8e4f9ca0c774a9921ea4de` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.047668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9754582e3f8e4f9ca0c774a9921ea4de)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.049724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.055037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fdd8e-0d95-4026-886a-2e315a0fa45b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.068599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.070175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.070623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.070931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.071794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:04.074941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.078689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.102449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 835/1648] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:04.108921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.134476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6118-6003-40b9-b373-642e9aa731d7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.146957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.147793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.148323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.148822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.149536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.151725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.271675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 836/1649] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:04 2022] POST /identity/v3/projects => generated 389 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:04.272787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.356571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11e43c-480d-48dc-ad67-0b08cb634970 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.366851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.368213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.368935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.369364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.370217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.372623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.396578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9754582e3f8e4f9ca0c774a9921ea4de'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:04.405645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9754582e3f8e4f9ca0c774a9921ea4de'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:04.406395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9754582e3f8e4f9ca0c774a9921ea4de'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:04.462523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 837/1650] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:04 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:04.467743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.480285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9754582e3f8e4f9ca0c774a9921ea4de'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:04.481362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9754582e3f8e4f9ca0c774a9921ea4de'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:04.488351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181d128-05f1-4351-a794-d2443bd58a4e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.499736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.500598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.501152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.501446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.502017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.504230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.510473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 814/1651] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:03 2022] DELETE /identity/v3/users/9754582e3f8e4f9ca0c774a9921ea4de => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:04.526726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 838/1652] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:04.527959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.530988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:04.595690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:04.626088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769909-ebb2-4fc7-b05f-21a69097b826 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.632179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.632905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.633252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.633541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.634205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.636785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425ea69-f48b-4724-b2a0-b2c422a4dac9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.653182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.660220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 815/1653] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:04.666117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4e0b8-3bf8-4502-a231-9f67263b2c89 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.684035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.684783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.685072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.685372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] PATH_INFO: `/v3/projects/024ef929d92e45ada9e834f01047b52c/users/2d81bc4793a341e3bf7f5e726ea0f187/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.705608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=024ef929d92e45ada9e834f01047b52c, user_id=2d81bc4793a341e3bf7f5e726ea0f187, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.708818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:04.777228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 839/1654] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:04 2022] PUT /identity/v3/projects/024ef929d92e45ada9e834f01047b52c/users/2d81bc4793a341e3bf7f5e726ea0f187/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:04.790858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.792625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.792625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.832230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] MFA Rules not processed for user `2d81bc4793a341e3bf7f5e726ea0f187`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:04.833734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:04.906698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 816/1655] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:04.911351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:04.979218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1665413-9a50-4be3-b134-1c168f73dffc admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:04.984042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:04.984650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:04.984933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:04.985207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:04.985873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:04.987829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.043952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.043952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.063322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bc26eb-ddf6-4315-83e3-283a08adc266 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.068932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.069587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.069865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.070139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] PATH_INFO: `/v3/projects/547bac1868064923b0e9096834a7f0e2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.071326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=547bac1868064923b0e9096834a7f0e2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.073287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.074354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 840/1656] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:04 2022] POST /identity/v3/projects => generated 389 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:05.079064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.141722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbde31-119c-453d-b3d8-6e945f44b942 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.146602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.147251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.147529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.147806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.148503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.150413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.194290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:05.195571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '547bac1868064923b0e9096834a7f0e2', 'request_id': 'req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:05.202738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '547bac1868064923b0e9096834a7f0e2', 'request_id': 'req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:05.208347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '547bac1868064923b0e9096834a7f0e2', 'request_id': 'req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:05.236844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 841/1657] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:05 2022] POST /identity/v3/users => generated 301 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:05.237900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.247884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0a212-91e1-4312-b367-b21d7473ab8b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.314703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.315998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.316164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.316438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.318615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.318728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.319757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] The token cache is being invalidate because project 547bac1868064923b0e9096834a7f0e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:05.320276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:05.321104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e8bb0f31-592d-4ea6-8be4-f8c95c4da196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:05.323985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 817/1658] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:05 2022] DELETE /identity/v3/projects/547bac1868064923b0e9096834a7f0e2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:05.329021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1af94f-b77a-4009-a6aa-4af74339f445 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.329314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1af94f-b77a-4009-a6aa-4af74339f445 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.329590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1af94f-b77a-4009-a6aa-4af74339f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.332904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 842/1659] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:05.336875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.339473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.352776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a121c1-9194-4cb8-aa0f-5286a5602ea3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.357732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.358471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.358740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.359053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] PATH_INFO: `/v3/projects/862f1966d1774122992a8b03630686fa/users/430f8ff6a426486d94d45c7f9241e90c/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.376698 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862f1966d1774122992a8b03630686fa, user_id=430f8ff6a426486d94d45c7f9241e90c, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.379454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.446323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 843/1660] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:05 2022] PUT /identity/v3/projects/862f1966d1774122992a8b03630686fa/users/430f8ff6a426486d94d45c7f9241e90c/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:05.450702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.474272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cd1af94f-b77a-4009-a6aa-4af74339f445 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:05.475608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cd1af94f-b77a-4009-a6aa-4af74339f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.554003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 818/1661] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:05.562560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.563168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.563751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.576008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501b3c8-769c-4a9d-9e02-fa341f4ae3d7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.582141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.582753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.583056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.583330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.583971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.584693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:05.585557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.585914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.594149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 819/1662] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:05.596290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 844/1663] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:05.598637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.600410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.608225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01a237-aa8c-4b0b-a19c-ae4a47c51ac7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.610430 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbfafd-de57-4938-aa4d-5dc6e52d652e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.615453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.616157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.616443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.616728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] PATH_INFO: `/v3/projects/862f1966d1774122992a8b03630686fa/users/430f8ff6a426486d94d45c7f9241e90c/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.618765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=862f1966d1774122992a8b03630686fa, user_id=430f8ff6a426486d94d45c7f9241e90c, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.621846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcb8a64-1608-4442-9a87-5d54b0ba3ccc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.623679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.623829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.625368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.625368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.625368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:05.625368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.626996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.642870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 820/1664] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:05.646098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.647544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 845/1665] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:05 2022] PUT /identity/v3/projects/862f1966d1774122992a8b03630686fa/users/430f8ff6a426486d94d45c7f9241e90c/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:05.656529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de6ad-b310-4209-8a93-ba04ae71af27 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.661248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.662394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.662987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.680643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:05.702426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] MFA Rules not processed for user `430f8ff6a426486d94d45c7f9241e90c`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:05.702426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.723517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.724132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.724409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.728491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.728491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.728491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.781339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 846/1666] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:05.796962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:05.798366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:05.869943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639b533-3832-4ebb-a932-2ee6dbfbc5f0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:05.873701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:05.874313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:05.874590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:05.874883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:05.887071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:05.888615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:05.900083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 847/1667] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:06.002762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 821/1668] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:05 2022] POST /identity/v3/projects => generated 383 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:06.005605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.020373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078116-043f-4410-8861-fba8d703ff6e service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.043922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.049699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:06.081661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.082274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.082553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.082851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.083475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.085423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.108921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43913c50-3bdf-402b-8ee0-713d3b6077f8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.113476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.114100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.114376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.114645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] PATH_INFO: `/v3/users/2d81bc4793a341e3bf7f5e726ea0f187` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.115763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d81bc4793a341e3bf7f5e726ea0f187)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.117697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.199810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 848/1669] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:06 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:06.204281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.214376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f55a05-7838-4686-9edc-96932de4146b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.220308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.224031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.224031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.224031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.224031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.224031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.233601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 849/1670] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:06.237316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.246497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3766859-016e-4399-97e2-076a91ea06fc admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.251133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.251689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.251965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.252235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.252837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.254352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.265339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 850/1671] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:06.269436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.279121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a1f7f-ae31-47ba-b888-68e28f170110 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.284541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.285173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.285449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.285721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] PATH_INFO: `/v3/projects/55e369bdff7f4126afd337f5f33de561/users/609d747eee294969b7a602d2711131ca/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.293030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:06.306978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e369bdff7f4126afd337f5f33de561, user_id=609d747eee294969b7a602d2711131ca, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.309782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fa64f-4521-4e26-b572-5da01c89308c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.322559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d81bc4793a341e3bf7f5e726ea0f187'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.331206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d81bc4793a341e3bf7f5e726ea0f187'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.331939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d81bc4793a341e3bf7f5e726ea0f187'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.365596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:06.388588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d81bc4793a341e3bf7f5e726ea0f187'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.389989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9b2b453b-419b-4be0-91e1-cac6be24da30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d81bc4793a341e3bf7f5e726ea0f187'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.407144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 851/1672] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:06 2022] PUT /identity/v3/projects/55e369bdff7f4126afd337f5f33de561/users/609d747eee294969b7a602d2711131ca/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:06.409000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.409287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.409552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.419410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 822/1673] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:06 2022] DELETE /identity/v3/users/2d81bc4793a341e3bf7f5e726ea0f187 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:06.427881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.428218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.428492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.447554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] MFA Rules not processed for user `609d747eee294969b7a602d2711131ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:06.453674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:06.455382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:06.458298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:06.537485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 823/1674] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:06.548178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.549542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:06.549970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 852/1675] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:06.556014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.557314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:06.571920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd528c-a1c3-4661-9d23-03814a4da450 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.577060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.578301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.578586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.578894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.579948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.582229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.619770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43fbfb-b718-473e-b978-eb0ac013d61d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.624137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.624752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.625031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.625578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.634349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.635865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.646113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 824/1676] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:06.646956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.648434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:06.662270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128910-3f24-484d-8035-0af4ca5a9e07 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.667769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.668683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.669121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 853/1677] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:06 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:06.672683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.672999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] PATH_INFO: `/v3/users/25d27df2e39b48cbba40a65eec1cbc60` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.673156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.674865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d27df2e39b48cbba40a65eec1cbc60)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.677949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.751842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff7fcb-500d-4df3-8c67-3e8b41e67ced admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.756817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.757429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.757850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.758405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.759077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.763423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.883670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 854/1678] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:06 2022] POST /identity/v3/users => generated 298 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:06.885130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.894893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4418a05-93fa-4614-ab45-71dfad47693b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.899761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.901147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.901147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.901147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.901625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.903308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.917987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 855/1679] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:06.923889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:06.935692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bef661-ce4a-4fc2-8583-a9434f606a24 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:06.941056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:06.947137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:06.947137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:06.947137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] PATH_INFO: `/v3/projects/8beea93972b44bb4a6b634c57f441146/users/b04478fc290a4e76aee4fad5c23b5199/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:06.958548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8beea93972b44bb4a6b634c57f441146, user_id=b04478fc290a4e76aee4fad5c23b5199, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:06.961349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:06.980978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d27df2e39b48cbba40a65eec1cbc60'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.988188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d27df2e39b48cbba40a65eec1cbc60'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:06.992907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d27df2e39b48cbba40a65eec1cbc60'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.086042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d27df2e39b48cbba40a65eec1cbc60'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.087636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:07.090673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52e618b6-dfd9-491b-ae58-0a94ea2c5260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d27df2e39b48cbba40a65eec1cbc60'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.091967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 856/1680] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:06 2022] PUT /identity/v3/projects/8beea93972b44bb4a6b634c57f441146/users/b04478fc290a4e76aee4fad5c23b5199/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:07.096519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.098080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.110908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 825/1681] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:06 2022] DELETE /identity/v3/users/25d27df2e39b48cbba40a65eec1cbc60 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:07.116823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.117114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.117391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.137022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:07.138045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.180177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d3d65-3705-4521-a954-596c882bdab0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.210547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 826/1682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:07.211441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.212699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.224602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f583b-2d08-4a47-bcd1-f9e6dc17306d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.229163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.229774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.230049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.230318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] PATH_INFO: `/v3/projects/024ef929d92e45ada9e834f01047b52c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.231421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=024ef929d92e45ada9e834f01047b52c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.233330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.238708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:07.243142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.244626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.244626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.244626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.244911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.246486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.260006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 857/1683] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:07.260006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.260006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.325773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a03f-e2c9-4fbb-a573-c861ced96a9e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.330306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.330987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.331267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.331539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.340901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.342379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.351952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 858/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:07.352782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.362741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024ef929d92e45ada9e834f01047b52c', 'request_id': 'req-a36a6917-99b0-4a18-a09b-b600c40c8b0a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.363823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8da2-916b-41bc-8c52-52beadf393f9 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.370012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024ef929d92e45ada9e834f01047b52c', 'request_id': 'req-a36a6917-99b0-4a18-a09b-b600c40c8b0a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.377455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024ef929d92e45ada9e834f01047b52c', 'request_id': 'req-a36a6917-99b0-4a18-a09b-b600c40c8b0a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.422273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.422857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.423137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.423405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] PATH_INFO: `/v3/projects/8beea93972b44bb4a6b634c57f441146/users/b04478fc290a4e76aee4fad5c23b5199/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.425395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=8beea93972b44bb4a6b634c57f441146, user_id=b04478fc290a4e76aee4fad5c23b5199, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.428111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f448622-568d-41c3-b5eb-8ded677428ad admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.441976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 859/1685] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:07 2022] PUT /identity/v3/projects/8beea93972b44bb4a6b634c57f441146/users/b04478fc290a4e76aee4fad5c23b5199/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:07.453018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.458292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.458292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.459049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] The token cache is being invalidate because project 024ef929d92e45ada9e834f01047b52c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:07.459049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.459049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.460800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 827/1686] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:07 2022] DELETE /identity/v3/projects/024ef929d92e45ada9e834f01047b52c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:07.464600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.465960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.479675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6917-99b0-4a18-a09b-b600c40c8b0a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.484176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] MFA Rules not processed for user `b04478fc290a4e76aee4fad5c23b5199`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:07.485385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.540231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.540885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.545509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.545509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] PATH_INFO: `/v3/users/430f8ff6a426486d94d45c7f9241e90c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.545509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=430f8ff6a426486d94d45c7f9241e90c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.545509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.569594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 860/1687] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:07.583054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.584557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.649834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d731c-1541-4b93-a5b7-7ffcc8f0313b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.652834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:07.659734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.660610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.660899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.662999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.673807 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.675314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:07.687175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 861/1688] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:07.704080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:07.755153 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430f8ff6a426486d94d45c7f9241e90c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.763883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430f8ff6a426486d94d45c7f9241e90c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.763883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430f8ff6a426486d94d45c7f9241e90c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.851783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430f8ff6a426486d94d45c7f9241e90c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.852548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430f8ff6a426486d94d45c7f9241e90c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:07.872174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 828/1689] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:07 2022] DELETE /identity/v3/users/430f8ff6a426486d94d45c7f9241e90c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:07.872696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.882106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422371a7-72f2-467b-ae9e-3861d7c0baab service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.897066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:07.898937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.949862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.950485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.950772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.951126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.952991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:07.980171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8939-1480-4ff8-93a1-c24cba094a50 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:07.985641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:07.986276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:07.986555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:07.986853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] PATH_INFO: `/v3/projects/0bcb9273129e4bdf9c513508485a4003` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:07.987963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bcb9273129e4bdf9c513508485a4003)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:07.989874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.022552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.024034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.033021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 862/1690] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:08.100079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bcb9273129e4bdf9c513508485a4003', 'request_id': 'req-16c03cbc-64f2-43e6-b76e-dad672b1a575'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.106768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bcb9273129e4bdf9c513508485a4003', 'request_id': 'req-16c03cbc-64f2-43e6-b76e-dad672b1a575'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.114713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bcb9273129e4bdf9c513508485a4003', 'request_id': 'req-16c03cbc-64f2-43e6-b76e-dad672b1a575'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.198633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] The token cache is being invalidate because project 0bcb9273129e4bdf9c513508485a4003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:08.201926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.201926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.201926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 829/1691] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:07 2022] DELETE /identity/v3/projects/0bcb9273129e4bdf9c513508485a4003 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:08.205981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.217918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352e4ef-2b21-46da-8538-5ce55907a136 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.277769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.278409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.278693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.279016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] PATH_INFO: `/v3/users/4d785352f90046c8a9a3060873899cf6` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.281656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d785352f90046c8a9a3060873899cf6)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.284232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.316730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.318531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:08.346004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c03cbc-64f2-43e6-b76e-dad672b1a575 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.350676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.351346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.351623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.351892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] PATH_INFO: `/v3/projects/862f1966d1774122992a8b03630686fa` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.352993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=862f1966d1774122992a8b03630686fa)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.354930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.375074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:08.460142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d785352f90046c8a9a3060873899cf6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.463226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862f1966d1774122992a8b03630686fa', 'request_id': 'req-3c601c43-2d4f-4f64-ad57-063d67fd10cc'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.467979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d785352f90046c8a9a3060873899cf6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.468664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d785352f90046c8a9a3060873899cf6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.470277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862f1966d1774122992a8b03630686fa', 'request_id': 'req-3c601c43-2d4f-4f64-ad57-063d67fd10cc'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.476085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862f1966d1774122992a8b03630686fa', 'request_id': 'req-3c601c43-2d4f-4f64-ad57-063d67fd10cc'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.554169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d785352f90046c8a9a3060873899cf6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.554787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d785352f90046c8a9a3060873899cf6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.578637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 863/1692] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:08 2022] DELETE /identity/v3/users/4d785352f90046c8a9a3060873899cf6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:08.579529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.584941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:08.640733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] The token cache is being invalidate because project 862f1966d1774122992a8b03630686fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:08.641059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.641746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c601c43-2d4f-4f64-ad57-063d67fd10cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.643590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 830/1693] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:08 2022] DELETE /identity/v3/projects/862f1966d1774122992a8b03630686fa => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:08.645222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af93a1-4b07-46ba-bff3-f9050fe026f8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.648552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dbec-6fbf-482c-816f-e0f614f643c2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.648931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dbec-6fbf-482c-816f-e0f614f643c2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.649219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843dbec-6fbf-482c-816f-e0f614f643c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.650046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.650697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.651014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.651294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] PATH_INFO: `/v3/users/609d747eee294969b7a602d2711131ca` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.652448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=609d747eee294969b7a602d2711131ca)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.654457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.671580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b843dbec-6fbf-482c-816f-e0f614f643c2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:08.673120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b843dbec-6fbf-482c-816f-e0f614f643c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:08.682655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 831/1694] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:08.691554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.691945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.692286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.712786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:08.714013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:08.730340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 832/1695] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:08.734206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.743808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8016ddf-383a-4384-a3ed-eb08baee9fdc None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.749489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.750202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.750538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.750927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.751640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:08.753774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.753774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.770220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 833/1696] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:08.772652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.784346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3935a1-903f-4572-b4c9-1c3edd953077 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.793727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.795010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.795332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.795647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.796348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.798576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:08.906095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:08.916793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609d747eee294969b7a602d2711131ca'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.919701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 834/1697] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:08 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:08.924090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:08.926997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609d747eee294969b7a602d2711131ca'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.928010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609d747eee294969b7a602d2711131ca'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:08.985753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b191a9-7ec6-4068-9412-49aec532bd33 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:08.990266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:08.990904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:08.991187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:08.991461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:08.992077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:08.993984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b3e2-ab10-4671-ae94-a3515c5d9bc4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.000920 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609d747eee294969b7a602d2711131ca'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.001409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609d747eee294969b7a602d2711131ca'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.022127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 864/1698] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:08 2022] DELETE /identity/v3/users/609d747eee294969b7a602d2711131ca => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:09.022127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.023779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.087728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09adc7-1ac8-49cc-9d73-4df4d1e10d2c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.092549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.093181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.093463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.093740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] PATH_INFO: `/v3/projects/fa8c810265104feba637c52c2ed6b47b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.094997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8c810265104feba637c52c2ed6b47b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.097205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.097611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 835/1699] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:08 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:09.098868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.099155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.099427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.119214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:09.120393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.128339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 836/1700] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:09.129107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.130300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.142072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a424f-f2db-4e23-befd-832a7d5b0d24 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.146599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.147208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.147483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.147749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.148403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.149955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.159415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 837/1701] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:09.160183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.162029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.193910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:09.202924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8c810265104feba637c52c2ed6b47b', 'request_id': 'req-76339001-0759-4949-b0a8-e1361bbf261f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.209181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8c810265104feba637c52c2ed6b47b', 'request_id': 'req-76339001-0759-4949-b0a8-e1361bbf261f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.214684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8c810265104feba637c52c2ed6b47b', 'request_id': 'req-76339001-0759-4949-b0a8-e1361bbf261f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.219706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cc11-8510-4c08-b66a-ed12240ece5f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.267799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.268367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.268651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.268929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.279518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] The token cache is being invalidate because project fa8c810265104feba637c52c2ed6b47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:09.279978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.280709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.282256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 865/1702] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:09 2022] DELETE /identity/v3/projects/fa8c810265104feba637c52c2ed6b47b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:09.283212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.284609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.331788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.333442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e9476-b4ee-4af7-acf4-edfe0a8ad4e1 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.343450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 838/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:09.344806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48009f0-a80e-4ed5-ba8b-1bc4eb404913 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.345057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48009f0-a80e-4ed5-ba8b-1bc4eb404913 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.355176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48009f0-a80e-4ed5-ba8b-1bc4eb404913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.355601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339001-0759-4949-b0a8-e1361bbf261f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.356522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.357142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.357423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.357696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.358899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.360773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.366767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c48009f0-a80e-4ed5-ba8b-1bc4eb404913 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:09.368139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c48009f0-a80e-4ed5-ba8b-1bc4eb404913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.376084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 866/1704] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:09.378600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 839/1705] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:09.379266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.388998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.389406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.390250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.395021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe43fb-e136-4a56-afb1-de168e2f2530 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.402431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.406707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.406707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.406707 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] PATH_INFO: `/v3/projects/3f965e23cb374b20b01b4c4c1f08c27b/users/0c4707f165934f15893df0caf904ea37/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.412098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:09.413318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.422294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f965e23cb374b20b01b4c4c1f08c27b, user_id=0c4707f165934f15893df0caf904ea37, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.422602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 840/1706] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:09.425518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fc607-70b3-4f28-87fa-e7568735c955 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.432444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.443284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723a3b2-ae6b-4218-a94e-c9e3cf24990f None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.448172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.448779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.449065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.449335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.449981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:09.450382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.452151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.463928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 841/1707] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:09.468323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.478189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff58e52-6a7e-40ec-8198-10bf7435d0ee admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.485736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 867/1708] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:09 2022] PUT /identity/v3/projects/3f965e23cb374b20b01b4c4c1f08c27b/users/0c4707f165934f15893df0caf904ea37/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:09.496162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.496454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.496734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.526160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] MFA Rules not processed for user `0c4707f165934f15893df0caf904ea37`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:09.527398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.541518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.542532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.542995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.543416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.544465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.548207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.598602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 868/1709] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:09.610446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.612007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.635095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 842/1710] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:09.639789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.674134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaebfec-85b2-4971-9210-547b12c6ca44 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.685671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:09.705775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b56bd-45d0-43cd-a13c-07954adb18db admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.710629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.711349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.711630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.711900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.712521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.714612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.726547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.727189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.727469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.727743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.797830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.799554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.801047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:09.808562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 843/1711] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:09 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:09.809564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.810437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 869/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:09.812719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.814615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:09.823108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277629-e887-4b4c-a206-b1760ca22628 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.828376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.829004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.829309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.829577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.830212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.832020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.835583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614baa2-143f-4fd7-b99b-72ce1de5f9f3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.840376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.842204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.842204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.842204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] PATH_INFO: `/v3/projects/55e369bdff7f4126afd337f5f33de561` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.842737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e369bdff7f4126afd337f5f33de561)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.842995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 870/1713] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:09.843919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:09.844707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.853398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f728a11-f908-4dec-aa29-c75ed68d3cd5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:09.857975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:09.858609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:09.858919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:09.859196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] PATH_INFO: `/v3/users/0c4707f165934f15893df0caf904ea37` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:09.860320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4707f165934f15893df0caf904ea37)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:09.862205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:09.975444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e369bdff7f4126afd337f5f33de561', 'request_id': 'req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.982335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e369bdff7f4126afd337f5f33de561', 'request_id': 'req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:09.989962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e369bdff7f4126afd337f5f33de561', 'request_id': 'req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.067980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:10.104092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] The token cache is being invalidate because project 55e369bdff7f4126afd337f5f33de561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:10.104494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.105140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.110097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 844/1714] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:09 2022] DELETE /identity/v3/projects/55e369bdff7f4126afd337f5f33de561 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:10.111478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.114012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.115304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4707f165934f15893df0caf904ea37'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.124757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4707f165934f15893df0caf904ea37'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.125463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4707f165934f15893df0caf904ea37'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.178109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451fc54-ec31-4ea7-a876-cd34e87c8ae4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.182732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.183378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.183661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.183935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.184567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.187080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:10.188797 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4707f165934f15893df0caf904ea37'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.189441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4707f165934f15893df0caf904ea37'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.197932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 845/1715] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:10.198781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.202617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.207913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 871/1716] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:09 2022] DELETE /identity/v3/users/0c4707f165934f15893df0caf904ea37 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:10.208700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.209998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.215334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939d2bc-6231-4cf3-a3d0-e42755b8da29 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.264905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:10.273932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.274566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.274850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16d57-5b53-4372-864e-ae70f3ed86a7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.274975 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.275140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] PATH_INFO: `/v3/users/b04478fc290a4e76aee4fad5c23b5199` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.276288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04478fc290a4e76aee4fad5c23b5199)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.278286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:10.279321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.279970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.280244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.280519 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] PATH_INFO: `/v3/projects/7583fdcc74fd46cf9cb36379c9cb17ba/users/f293b0f5c0f24693af5dcec021c71d50/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.295043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7583fdcc74fd46cf9cb36379c9cb17ba, user_id=f293b0f5c0f24693af5dcec021c71d50, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.298656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80184580-5904-463a-89e0-11000efb43b2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:10.309306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:10.354640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 872/1717] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:10 2022] PUT /identity/v3/projects/7583fdcc74fd46cf9cb36379c9cb17ba/users/f293b0f5c0f24693af5dcec021c71d50/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:10.355879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5793c-e044-44e1-a2ab-c4c7974580b8 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.356158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5793c-e044-44e1-a2ab-c4c7974580b8 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.356447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5793c-e044-44e1-a2ab-c4c7974580b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.375505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8ed5793c-e044-44e1-a2ab-c4c7974580b8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:10.376711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8ed5793c-e044-44e1-a2ab-c4c7974580b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.435329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 873/1718] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:10.436659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.436939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.437211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.463602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] MFA Rules not processed for user `f293b0f5c0f24693af5dcec021c71d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:10.464801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.478434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04478fc290a4e76aee4fad5c23b5199'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.485963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04478fc290a4e76aee4fad5c23b5199'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.486715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04478fc290a4e76aee4fad5c23b5199'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.528816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 874/1719] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:10.529649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.531490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.569941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04478fc290a4e76aee4fad5c23b5199'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.570606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04478fc290a4e76aee4fad5c23b5199'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:10.586399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c59b0e-c2b6-4d31-abbd-0d898ab87fa7 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.588233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 846/1720] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:10 2022] DELETE /identity/v3/users/b04478fc290a4e76aee4fad5c23b5199 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:10.589034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.590335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.638738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.641983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.641983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.641983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.641983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.667189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c09dd-c1df-49fc-8861-540afa7cf725 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.671796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.672753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.673112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.673453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.675301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.711158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.712612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:10.728148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 875/1721] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:10.747543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.749973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:10.752002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:10.761714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 847/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:10.785297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:10.786746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:10.797902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:10.801792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803e336-86a0-44ee-8d40-311490174433 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:10.807381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:10.808043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:10.808381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:10.808654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] PATH_INFO: `/v3/users/f293b0f5c0f24693af5dcec021c71d50` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:10.809795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f293b0f5c0f24693af5dcec021c71d50)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:10.811878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.031724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.034611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.049038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f293b0f5c0f24693af5dcec021c71d50'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.050658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c08a4-1166-46b6-a03c-191caa29986b service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.058158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.058158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.058158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.058158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] PATH_INFO: `/v3/projects/8beea93972b44bb4a6b634c57f441146` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.059032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8beea93972b44bb4a6b634c57f441146)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.060994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.070183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f293b0f5c0f24693af5dcec021c71d50'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.071109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f293b0f5c0f24693af5dcec021c71d50'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.143203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f293b0f5c0f24693af5dcec021c71d50'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.143770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-81d27c33-db97-4704-a8e9-dd921c406f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f293b0f5c0f24693af5dcec021c71d50'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.161870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 876/1723] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:10 2022] DELETE /identity/v3/users/f293b0f5c0f24693af5dcec021c71d50 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:11.166858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.167223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.167581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.191895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:11.193321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.199765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8beea93972b44bb4a6b634c57f441146', 'request_id': 'req-b08647f0-9dd3-4e21-86be-983b36f52e67'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.208386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8beea93972b44bb4a6b634c57f441146', 'request_id': 'req-b08647f0-9dd3-4e21-86be-983b36f52e67'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.223615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8beea93972b44bb4a6b634c57f441146', 'request_id': 'req-b08647f0-9dd3-4e21-86be-983b36f52e67'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.257920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:11.274717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 877/1724] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:11.279712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.280831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.292757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eedc6a-22ba-48ec-b03d-6003d3afd16d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.296336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:11.298219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] The token cache is being invalidate because project 8beea93972b44bb4a6b634c57f441146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:11.298613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.299263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.300842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 848/1725] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:11 2022] DELETE /identity/v3/projects/8beea93972b44bb4a6b634c57f441146 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:11.301776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.302427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.302709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.303013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] PATH_INFO: `/v3/projects/3f965e23cb374b20b01b4c4c1f08c27b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.304454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f965e23cb374b20b01b4c4c1f08c27b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.306318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.306691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.307849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.367659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08647f0-9dd3-4e21-86be-983b36f52e67 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.371325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.372924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.373147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.373429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.375753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.387351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.388776 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1d279-15b3-4f01-bcd9-7915a50230d1 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.400172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 849/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:11.400466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a6ee-b90c-4b7d-a6c6-7fde75e8246f None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.400758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a6ee-b90c-4b7d-a6c6-7fde75e8246f None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.401038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338a6ee-b90c-4b7d-a6c6-7fde75e8246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.416858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f965e23cb374b20b01b4c4c1f08c27b', 'request_id': 'req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.424434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f965e23cb374b20b01b4c4c1f08c27b', 'request_id': 'req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.424904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c338a6ee-b90c-4b7d-a6c6-7fde75e8246f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:11.428237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c338a6ee-b90c-4b7d-a6c6-7fde75e8246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.432149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f965e23cb374b20b01b4c4c1f08c27b', 'request_id': 'req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.492682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 850/1727] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:11.501744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.502030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.502301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.520200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] The token cache is being invalidate because project 3f965e23cb374b20b01b4c4c1f08c27b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:11.520487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.521244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7fe00be5-1f65-40a9-93f9-b6c6d5a0dc00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:11.522184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:11.523823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 878/1728] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:11 2022] DELETE /identity/v3/projects/3f965e23cb374b20b01b4c4c1f08c27b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:11.523929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.527577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.527866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.528141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.532184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 851/1729] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:11.536300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.545589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57198a1b-5fcb-4e00-9ccc-46006e359e2c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.551102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.551234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:11.551744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.552029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.552306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.552541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.552951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:11.553258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.554876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ba06d-f19e-4373-bdfc-08a29f9df0bd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.560876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 879/1730] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:11.567181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 852/1731] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:11.572424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.575587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.575881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.576198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.582431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2823-06be-446c-a92c-af36639a061e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.587166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.587815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.588087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.588375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.589013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.591208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.596279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:11.597885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.611052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 853/1732] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:11.612650 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.621853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bbbaaf-8147-4828-ac9d-541087d4ac86 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.627230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.627837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.628112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.632509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.632509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:11.632509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.632509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.643462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 854/1733] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:11.647756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.657692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2ed0d-78d0-4af3-adf5-eed39918eab2 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.662627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.663391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.663669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.663941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.664562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.666462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.687279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 880/1734] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:11.688470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.750742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:11.758431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fde0ce-f8c6-4c54-8c11-b7993aae706b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.764107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.764761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.765297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.765577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.766228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.768192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.774081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 855/1735] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:11.777988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.839361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b66f8-7319-4ac6-9bc8-3d971d231553 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.844736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.845350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.845633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.845905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.846528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.848692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.866010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:11.871249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 881/1736] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:11.871249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.874913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:11.886060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cd89b-a4ff-4724-b1ef-844204efe8fa admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.890634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.891276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.891557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.891832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] PATH_INFO: `/v3/projects/7583fdcc74fd46cf9cb36379c9cb17ba` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.892950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7583fdcc74fd46cf9cb36379c9cb17ba)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.894905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.930256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 856/1737] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:11 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:11.930256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.940545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e37ab-e94c-4156-a8e4-0bb85469d055 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.945119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.945962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.946096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.946328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.947429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.949352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.959877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 857/1738] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:11.960805 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:11.971887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475de84f-c259-4896-9ff9-2e9fb67cee11 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:11.976571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:11.977142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:11.977418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:11.977689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:11.978304 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:11.979924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:11.992368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 858/1739] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:11.993300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.003095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3148-3613-4827-a376-ec33033a05b9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.016167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7583fdcc74fd46cf9cb36379c9cb17ba', 'request_id': 'req-6cff96b6-67c2-48e0-85db-12764059d445'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:12.024568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7583fdcc74fd46cf9cb36379c9cb17ba', 'request_id': 'req-6cff96b6-67c2-48e0-85db-12764059d445'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:12.030059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7583fdcc74fd46cf9cb36379c9cb17ba', 'request_id': 'req-6cff96b6-67c2-48e0-85db-12764059d445'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:12.062405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.063148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.063433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.063710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.064347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.066048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.078648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 859/1740] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:12.078648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.085720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca513a-1e7b-4e2e-b489-a363b9874dcb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.090317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.090956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.091242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.091520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.092128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.094167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.103979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 860/1741] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:12.104751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.109607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] The token cache is being invalidate because project 7583fdcc74fd46cf9cb36379c9cb17ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:12.111815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:12.111815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:12.112591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 882/1742] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:11 2022] DELETE /identity/v3/projects/7583fdcc74fd46cf9cb36379c9cb17ba => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:12.115424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae348330-6b2a-406e-9b2a-78cc3e02281d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.116721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.118150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.120170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.120810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.121097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.121429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] PATH_INFO: `/v3/projects/74c1dc5bddae439a91429fd9a5dd9981/users/61fb9881b7d84f52bae770003d6107b7/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.131802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff96b6-67c2-48e0-85db-12764059d445 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.139803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.139953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c1dc5bddae439a91429fd9a5dd9981, user_id=61fb9881b7d84f52bae770003d6107b7, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.140064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.140064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.140064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] PATH_INFO: `/v3/projects/b7eb8dbf3ca14a07af0d590ca06f5c90/users/1ce4b77ecda442b8bf3938ae40635c68/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.140361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb57116-4ef0-4f70-8e65-3467f8965c04 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.152330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7eb8dbf3ca14a07af0d590ca06f5c90, user_id=1ce4b77ecda442b8bf3938ae40635c68, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.154952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ffa4d-2557-4b02-ba2d-c95f55af9f3a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.261669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 861/1743] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:12 2022] PUT /identity/v3/projects/74c1dc5bddae439a91429fd9a5dd9981/users/61fb9881b7d84f52bae770003d6107b7/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:12.263207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50662f2-20c7-4501-9ac3-ac2f05012844 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.263766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50662f2-20c7-4501-9ac3-ac2f05012844 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.264049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50662f2-20c7-4501-9ac3-ac2f05012844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.268591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:12.271275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 883/1744] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:12 2022] PUT /identity/v3/projects/b7eb8dbf3ca14a07af0d590ca06f5c90/users/1ce4b77ecda442b8bf3938ae40635c68/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:12.272606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e68996-f4cd-41cd-a2ce-9a934a540d74 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.272891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e68996-f4cd-41cd-a2ce-9a934a540d74 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.273158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e68996-f4cd-41cd-a2ce-9a934a540d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.295204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d50662f2-20c7-4501-9ac3-ac2f05012844 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:12.296421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d50662f2-20c7-4501-9ac3-ac2f05012844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.301890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a1e68996-f4cd-41cd-a2ce-9a934a540d74 None None] MFA Rules not processed for user `61fb9881b7d84f52bae770003d6107b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:12.303100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a1e68996-f4cd-41cd-a2ce-9a934a540d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.358253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 862/1745] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:12.359166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.359440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.359821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.381856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 884/1746] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:12.390382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] MFA Rules not processed for user `1ce4b77ecda442b8bf3938ae40635c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:12.391982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.397677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.397956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.398234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.426550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:12.427943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.437040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 885/1747] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:12.437838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.439424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.461096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beacb54-f297-4f7c-a3c2-1038154fef2b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.466585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.467432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.467857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.468878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.469997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.472818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.476712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 863/1748] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:12.477946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.495114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7658390-da4a-46a0-859c-38929ae2e089 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.500926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.501547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.501825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.502420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.502735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:12.503066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.504707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.524390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 864/1749] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:12.524390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.524390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:12.583322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ebae7-f57c-4759-b23f-17ac289cd968 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.587233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.587892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.588171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.588452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.597201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.598744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.599660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:12.607999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 865/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:12.610123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.622263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 886/1751] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:12.627626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.629375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39aad-7f54-4cec-94fa-8ac18c0548ea service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.694438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.695082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.695369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.695630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.695779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868c15b-acc3-4b70-9e4f-17dd502a61dd admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.696361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.698337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.701116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.701833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.702291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.703066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.703808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.705738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.854579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:12.863644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 887/1752] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:12.864842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.869002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 866/1753] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/projects => generated 377 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:12.870352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.880515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0163375-eb63-4909-b571-557bd094dc3d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.953589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.954202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.954478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.954752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.955957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.957939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.959925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767576-9db3-4ce4-83f1-61fb9ca5ca09 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.964693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.965326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.965803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.966107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:12.966720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:12.971381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 888/1754] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:12.973219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:12.973361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:12.982579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769bdf9d-98f9-4f46-8c26-620311fbdc86 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:12.987923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:12.988575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:12.988853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:12.997370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] PATH_INFO: `/v3/projects/453155968c6a43909df5ebdf51d77840/users/59213601b32647f1b6a66f6da78b32a6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.005967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=453155968c6a43909df5ebdf51d77840, user_id=59213601b32647f1b6a66f6da78b32a6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.020256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.079402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 867/1755] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:12 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:13.083219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.093388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46ee81-84bc-4f3d-9b21-1f2b514e62ac admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.099440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.100422 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.101048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.101472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.102200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.103956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.104472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 889/1756] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:12 2022] PUT /identity/v3/projects/453155968c6a43909df5ebdf51d77840/users/59213601b32647f1b6a66f6da78b32a6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:13.108475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.118447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 868/1757] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:13.122636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.133727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3ef7-9324-40dd-bcae-2d435627a278 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.177876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039c634-cb71-44c9-8df0-a9528802a77a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.182725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.183360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.183644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.183916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.184557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.186141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.194241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.194894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.195183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.195461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.196250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.196805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 890/1758] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:13.197944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.200758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.210800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42290f8f-dbcb-4c1e-86a3-8541ebd550d8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.216406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 869/1759] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:13.216545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.217043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.221445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.221445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] PATH_INFO: `/v3/projects/453155968c6a43909df5ebdf51d77840/users/59213601b32647f1b6a66f6da78b32a6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.221634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.221725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=453155968c6a43909df5ebdf51d77840, user_id=59213601b32647f1b6a66f6da78b32a6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.222626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7076f2-0695-406b-a849-6f29ec3acf5f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.229295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e75801-aa70-48f0-9a1c-5a0369cb171c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.235021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.235672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.235955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.236228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] PATH_INFO: `/v3/projects/924211a2eedf4c00ad49f2bcdf1c1431/users/0b22256920ae421f9af326d68d412375/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.239048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 891/1760] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:13 2022] PUT /identity/v3/projects/453155968c6a43909df5ebdf51d77840/users/59213601b32647f1b6a66f6da78b32a6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:13.249204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5eb6b-f7c4-4dd0-bd47-00cf55e8fbbd None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.250054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5eb6b-f7c4-4dd0-bd47-00cf55e8fbbd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.250381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5eb6b-f7c4-4dd0-bd47-00cf55e8fbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.255625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=924211a2eedf4c00ad49f2bcdf1c1431, user_id=0b22256920ae421f9af326d68d412375, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.258342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.278900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-12d5eb6b-f7c4-4dd0-bd47-00cf55e8fbbd None None] MFA Rules not processed for user `59213601b32647f1b6a66f6da78b32a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:13.280162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-12d5eb6b-f7c4-4dd0-bd47-00cf55e8fbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:13.354936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 892/1761] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:13.358691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 870/1762] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:13 2022] PUT /identity/v3/projects/924211a2eedf4c00ad49f2bcdf1c1431/users/0b22256920ae421f9af326d68d412375/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:13.369578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.369940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.370216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.407062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] MFA Rules not processed for user `0b22256920ae421f9af326d68d412375`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:13.407349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.407439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:13.439792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:13.462387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0798-e5fa-47dd-9d08-0634b21ca89e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.465685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.466389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.466958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.466958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.478001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 893/1763] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:13.487717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:13.489544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:13.546038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.548397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.567120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 871/1764] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:13.582479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685898f5-ad47-454d-87af-e9782bf66c0c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:13.585947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:13.587782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:13.587782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:13.587782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:13.600206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:13.601823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:13.613607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 894/1765] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:13.783402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:14.233752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:14.628836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:14.944579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:15.036029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:15.115706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:15.132763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:15.167204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:15.201147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:22.203950 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:22.239077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:22.269813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7cd07-6122-4a8b-a75f-96aa6927eb07 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:22.274593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:22.275257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.275536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.275808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] PATH_INFO: `/v3/users/1ce4b77ecda442b8bf3938ae40635c68` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.276927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce4b77ecda442b8bf3938ae40635c68)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:22.278874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:22.311409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfeb99-668a-49cd-8432-489846d12dca service glance] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:22.316466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:22.317099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.317377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.317647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] PATH_INFO: `/v3/users/61fb9881b7d84f52bae770003d6107b7` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.318769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61fb9881b7d84f52bae770003d6107b7)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:22.320751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:22.476492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce4b77ecda442b8bf3938ae40635c68'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.483670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce4b77ecda442b8bf3938ae40635c68'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.484351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce4b77ecda442b8bf3938ae40635c68'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.542475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61fb9881b7d84f52bae770003d6107b7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.550210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61fb9881b7d84f52bae770003d6107b7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.550927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61fb9881b7d84f52bae770003d6107b7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.551688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce4b77ecda442b8bf3938ae40635c68'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.552700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1ae0dfb8-57b7-4265-96a4-5d3d996e8f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce4b77ecda442b8bf3938ae40635c68'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.561885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:22.572522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 872/1766] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:22 2022] DELETE /identity/v3/users/1ce4b77ecda442b8bf3938ae40635c68 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:22.576984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1b8b-99e2-4d4f-92a5-367eb9c63fac None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.577270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1b8b-99e2-4d4f-92a5-367eb9c63fac None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.584674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1b8b-99e2-4d4f-92a5-367eb9c63fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.597380 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-247d1b8b-99e2-4d4f-92a5-367eb9c63fac None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:22.598592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-247d1b8b-99e2-4d4f-92a5-367eb9c63fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:22.640193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61fb9881b7d84f52bae770003d6107b7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.640902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61fb9881b7d84f52bae770003d6107b7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:22.658557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 873/1767] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:22.661088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 895/1768] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:22 2022] DELETE /identity/v3/users/61fb9881b7d84f52bae770003d6107b7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:22.665492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.666161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.666161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.674804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:22.674804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:22.691555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:22.693245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:22.745727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c26a6-ceba-4a73-aff6-43367b15e1f2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:22.749737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:22.750456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.750855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.751236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.765184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:22.769383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 874/1769] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:22.779003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:22.780460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:22.793012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be8141-645b-4a6d-805d-d981bacb0464 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:22.798249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:22.798893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:22.799189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:22.799704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:22.811338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:22.812518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:22.821861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:22.823531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:22.823644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 875/1770] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:22.833709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 896/1771] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:23.422537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:23.424236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:23.453128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:23.454811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d213e44-6a45-4db3-a1d8-5b20833f0d7f service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:23.455180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:23.461377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:23.462354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.462891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.463484 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] PATH_INFO: `/v3/projects/74c1dc5bddae439a91429fd9a5dd9981` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.465666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c1dc5bddae439a91429fd9a5dd9981)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:23.468964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb5981-a625-4d52-b138-6d3b6466c7f4 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:23.469155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:23.474874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:23.475511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.485227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.485227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] PATH_INFO: `/v3/projects/b7eb8dbf3ca14a07af0d590ca06f5c90` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.485227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7eb8dbf3ca14a07af0d590ca06f5c90)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:23.485227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:23.601748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c1dc5bddae439a91429fd9a5dd9981', 'request_id': 'req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.614259 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c1dc5bddae439a91429fd9a5dd9981', 'request_id': 'req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.620755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c1dc5bddae439a91429fd9a5dd9981', 'request_id': 'req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.634564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7eb8dbf3ca14a07af0d590ca06f5c90', 'request_id': 'req-a2c5c83b-3921-4d8d-97bb-c3791779ca14'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.641862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7eb8dbf3ca14a07af0d590ca06f5c90', 'request_id': 'req-a2c5c83b-3921-4d8d-97bb-c3791779ca14'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.648055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7eb8dbf3ca14a07af0d590ca06f5c90', 'request_id': 'req-a2c5c83b-3921-4d8d-97bb-c3791779ca14'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.701161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] The token cache is being invalidate because project 74c1dc5bddae439a91429fd9a5dd9981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:23.701993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.702144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.703673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 876/1772] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:23 2022] DELETE /identity/v3/projects/74c1dc5bddae439a91429fd9a5dd9981 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:23.708084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:23.709670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:23.714905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:23.786848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f5ce2-05c9-46b2-a16e-b9def3f0f8b5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:23.789113 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] The token cache is being invalidate because project b7eb8dbf3ca14a07af0d590ca06f5c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:23.789409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.790112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a2c5c83b-3921-4d8d-97bb-c3791779ca14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:23.791671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 897/1773] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:23 2022] DELETE /identity/v3/projects/b7eb8dbf3ca14a07af0d590ca06f5c90 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:23.796656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fd98-e49f-4858-9468-06a04d2d1c15 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.796941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fd98-e49f-4858-9468-06a04d2d1c15 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.797215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6fd98-e49f-4858-9468-06a04d2d1c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.816816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b9b6fd98-e49f-4858-9468-06a04d2d1c15 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:23.818504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b9b6fd98-e49f-4858-9468-06a04d2d1c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:23.860318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:23.861211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.861735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.862162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] PATH_INFO: `/v3/users/59213601b32647f1b6a66f6da78b32a6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.863332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59213601b32647f1b6a66f6da78b32a6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:23.865982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:23.904415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 898/1774] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:23.912255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.912543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.912816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.932509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:23.933831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:23.943602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 899/1775] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:23.947843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:23.957122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3d762-8ffe-48b2-9234-da9fdc783663 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:23.962351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:23.962971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:23.963251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:23.963525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:23.964162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:23.964466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:23.968096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:23.981068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 900/1776] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:23.985356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:23.995901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020b974-2f09-4a96-b621-2cf5aeca392a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.001385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.002017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.002318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.002606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.005477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.005820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.025699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:24.090475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59213601b32647f1b6a66f6da78b32a6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:24.097526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59213601b32647f1b6a66f6da78b32a6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:24.098223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59213601b32647f1b6a66f6da78b32a6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:24.116509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 901/1777] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:23 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:24.123690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.184361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59213601b32647f1b6a66f6da78b32a6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:24.185116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59213601b32647f1b6a66f6da78b32a6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:24.191634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef29c0-e074-4d01-a298-d93ce952984e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.196999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.197604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.197878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.198148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.198787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.201265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.218238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 877/1778] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:23 2022] DELETE /identity/v3/users/59213601b32647f1b6a66f6da78b32a6 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:24.302690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 902/1779] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:24 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:24.307056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.308638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:24.360625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:24.404317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07278c-bdbe-40b5-a9fb-f35a623da0d6 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.409473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.410125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.410404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.410934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.411625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.413400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.427178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 878/1780] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:24.438812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.454696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e083c-085b-4dcb-a79c-ca93991f4f45 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.464782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.465420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.465699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.465977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.466782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.468462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ca860-dab4-43b6-8ebd-88e0298d33a6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.486200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.494100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 903/1781] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:24.498973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e766-5c05-448e-a879-4aed1dc3e9f8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.506723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.507924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.508412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.508904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] PATH_INFO: `/v3/projects/df5f915d57114c8a8a4e376efbe3094c/users/30974466e60244b5993d5d79e4a48f3a/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.544959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5f915d57114c8a8a4e376efbe3094c, user_id=30974466e60244b5993d5d79e4a48f3a, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.547865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.621847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 879/1782] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:24 2022] PUT /identity/v3/projects/df5f915d57114c8a8a4e376efbe3094c/users/30974466e60244b5993d5d79e4a48f3a/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:24.634658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.635076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.635364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.675957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] MFA Rules not processed for user `30974466e60244b5993d5d79e4a48f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:24.677851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:24.768262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 904/1783] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:24.773924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.836554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5d3e2-549f-43a0-91b7-eb542f16ab37 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.841488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.842151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.842438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.842710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.843369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.845323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.909214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:24.910734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:24.926704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663066c-5b23-40dc-89c8-b19ca03f7d9b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:24.931356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:24.931975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:24.932252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:24.932523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] PATH_INFO: `/v3/projects/453155968c6a43909df5ebdf51d77840` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:24.933649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453155968c6a43909df5ebdf51d77840)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:24.935572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:24.971023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 880/1784] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:24 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:24.975769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.036694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71136b6-00f9-4ba3-b6fc-252017efd88d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.038517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453155968c6a43909df5ebdf51d77840', 'request_id': 'req-764aa0c6-2595-4faf-96fe-d4cfeea12637'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:25.045267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453155968c6a43909df5ebdf51d77840', 'request_id': 'req-764aa0c6-2595-4faf-96fe-d4cfeea12637'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:25.046523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:25.051790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453155968c6a43909df5ebdf51d77840', 'request_id': 'req-764aa0c6-2595-4faf-96fe-d4cfeea12637'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:25.066310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:25.094067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.096591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.096591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.096591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.096591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.097767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.120580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] The token cache is being invalidate because project 453155968c6a43909df5ebdf51d77840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:25.120869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:25.121528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-764aa0c6-2595-4faf-96fe-d4cfeea12637 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:25.123081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 905/1785] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:24 2022] DELETE /identity/v3/projects/453155968c6a43909df5ebdf51d77840 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:25.127816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0e8b-aa32-4745-a341-1a6956a9b2ec None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.128094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0e8b-aa32-4745-a341-1a6956a9b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.128366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0e8b-aa32-4745-a341-1a6956a9b2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.149339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-741f0e8b-aa32-4745-a341-1a6956a9b2ec None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:25.160229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-741f0e8b-aa32-4745-a341-1a6956a9b2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.160229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 906/1786] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:25.168184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.168464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.168822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.189018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:25.196701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.196843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 881/1787] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:24 2022] POST /identity/v3/users => generated 289 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:25.198867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 907/1788] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:25.199387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.201040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.202761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.212299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106ee1c-ee27-4661-8227-ba92ef999d60 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.213098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48828e3d-941a-4e02-b5ca-a1f8d1cbf72d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.217019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.217609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.217835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.217951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.218175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.218644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.218829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:25.218956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.219146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.219240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.219821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.220774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.221374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.232007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 882/1789] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:25.236538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 908/1790] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:25.236688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.237412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.245906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764184d-a545-490f-8de7-ad1fddaa790a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.247416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6b7e2-af34-4fe1-89d5-a0500be2223e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.250565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.251195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.251476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.251744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.251952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.252374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.252526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.252819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.253077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.253694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.253970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.255602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.264872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 883/1791] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:25.268921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.279129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaed396-25fe-4900-9c6d-34b57e37dd5f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.283687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.284309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.284584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.284853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] PATH_INFO: `/v3/projects/808b9a0b27e247868c3988871137e55e/users/e92ae010a33e4c778414f1354a4ba1f8/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.300287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808b9a0b27e247868c3988871137e55e, user_id=e92ae010a33e4c778414f1354a4ba1f8, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.302994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62de31-89b4-4ff2-8265-af637b7022f5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.318914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 909/1792] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:25.322986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.364106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 884/1793] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:25 2022] PUT /identity/v3/projects/808b9a0b27e247868c3988871137e55e/users/e92ae010a33e4c778414f1354a4ba1f8/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:25.374025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.374314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.374591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.386394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a54e-90e6-423e-9c06-d9696cfd22b2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.402264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] MFA Rules not processed for user `e92ae010a33e4c778414f1354a4ba1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:25.403500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.443727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.444335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.444619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.444929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.445558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.447552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.468646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 885/1794] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:25.473016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.483460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178dfbc-8158-48b1-a36c-0cb4689c39d4 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.487960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.488556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.488818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.491888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.491888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.491888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.550494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 910/1795] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:25.554493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.564075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c47f8-8ac1-4f30-84e5-b8854ae9dd3b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.568721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.569310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.569585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.569855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.570487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.573121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.586194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 886/1796] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:25.588015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 911/1797] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:25.590227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.593692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.607176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aaa49-afa8-47cc-8afc-b8fdaa31ede6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.626122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:25.655323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:25.671916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.672532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.672829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.673094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.674012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.675200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94714fb-f7e9-4525-a433-790701f99840 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.675935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.682854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.682854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.682854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.682854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.687589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.687589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a3a8e-225f-4240-a947-dbc1a30e36cf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.696743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 912/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:25.701996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.712658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddc231-5814-4fd4-a2a7-8d442442efe5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.717672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.718331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.725272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.725272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] PATH_INFO: `/v3/projects/a30976e8b35f436b84dc4488dcf5acaf/users/ce09c776cd77416091ea214b858afacd/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.735610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30976e8b35f436b84dc4488dcf5acaf, user_id=ce09c776cd77416091ea214b858afacd, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.738569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.825688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 913/1799] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:25 2022] PUT /identity/v3/projects/a30976e8b35f436b84dc4488dcf5acaf/users/ce09c776cd77416091ea214b858afacd/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:25.825902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 887/1800] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:25.830183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.830687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.831005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.831282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.852593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:25.853811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.891619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab8871-8add-4c01-9429-7b440a46ca63 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.896918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.897532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.897812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.898084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.898723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.900412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:25.910254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 914/1801] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:25.913004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 888/1802] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:25.914582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.923255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:25.924532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ed314-95d6-4da3-8475-3d27d6c6392a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:25.924720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:25.929943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:25.930581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:25.930963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:25.931245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] PATH_INFO: `/v3/projects/7747cb564ae44ddf8072d0fc81147938/users/124fb50cf80742b791d1cdb644a0672d/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:25.958746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7747cb564ae44ddf8072d0fc81147938, user_id=124fb50cf80742b791d1cdb644a0672d, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:25.961569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:26.003058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1b6f9-8cbe-478b-a1e6-6c636017aac8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:26.009911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:26.009911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:26.009911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:26.009911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:26.019588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:26.021751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:26.038271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 889/1803] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:26.071657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 915/1804] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:25 2022] PUT /identity/v3/projects/7747cb564ae44ddf8072d0fc81147938/users/124fb50cf80742b791d1cdb644a0672d/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:26.075757 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:26.087081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f4bcc-6854-4ba4-a62e-0263efa33ae8 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:26.159712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:26.160450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:26.160732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:26.161036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:26.161713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:26.163382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c232e-169b-4e32-a2ba-b84a7ceee311 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:26.174401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 890/1805] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:26.179295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:26.206194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc20af-19f0-4b2c-ac82-b2709bbc4337 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:26.223413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:26.224771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:26.225120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:26.225821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] PATH_INFO: `/v3/projects/7747cb564ae44ddf8072d0fc81147938/users/124fb50cf80742b791d1cdb644a0672d/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:26.229031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7747cb564ae44ddf8072d0fc81147938, user_id=124fb50cf80742b791d1cdb644a0672d, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:26.231705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:26.246507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 916/1806] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:26 2022] PUT /identity/v3/projects/7747cb564ae44ddf8072d0fc81147938/users/124fb50cf80742b791d1cdb644a0672d/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:26.259580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65216c-3eb7-435c-8954-ce2f6e2d0df5 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:26.259870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65216c-3eb7-435c-8954-ce2f6e2d0df5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:26.260145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65216c-3eb7-435c-8954-ce2f6e2d0df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:26.284924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:26.288182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3c65216c-3eb7-435c-8954-ce2f6e2d0df5 None None] MFA Rules not processed for user `124fb50cf80742b791d1cdb644a0672d`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:26.289472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3c65216c-3eb7-435c-8954-ce2f6e2d0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:26.364112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 891/1807] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:26.376441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:26.378719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:26.481579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88908493-f828-4834-a7a8-9ccf61109e2b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:26.485606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:26.486279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:26.486584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:26.486885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:26.488700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:26.499041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:26.552706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:26.554117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8a47a-52fc-4f05-a9bf-c706647ce889 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:26.564394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 917/1808] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:31.990253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:31.990813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:31.991229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:31.992949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 892/1809] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:19:32.001114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:32.001637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:32.001637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:32.023449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:32.024587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:32.102917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 918/1810] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:19:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:32.114653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:32.117032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:32.137486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadeaab-fdee-4f32-a4d1-193361873846 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:32.142790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:32.143458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:32.143742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:32.144017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:32.153361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:32.154995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:32.174397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:32.174773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 893/1811] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:32.179934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:32.286137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef795ab9-84d5-4c5d-af1c-e3855f8571a9 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:32.297252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:32.298003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:32.298382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:32.298737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] PATH_INFO: `/v3/users/fbbd63f10220465c996e6c706f1ef4a9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:32.300124 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbd63f10220465c996e6c706f1ef4a9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:32.302399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:32.513038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:32.519655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbd63f10220465c996e6c706f1ef4a9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:32.528578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbd63f10220465c996e6c706f1ef4a9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:32.529421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbd63f10220465c996e6c706f1ef4a9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:32.611016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbd63f10220465c996e6c706f1ef4a9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:32.612017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3a5d957d-5cc6-49c6-8e00-30e55e9137f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbd63f10220465c996e6c706f1ef4a9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:32.672170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 919/1812] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:32 2022] DELETE /identity/v3/users/fbbd63f10220465c996e6c706f1ef4a9 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:33.414618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:33.429742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11993a7-fdcd-4e26-8ede-873ecf3dc979 service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:33.591491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:33.596657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:33.596657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:33.596657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] PATH_INFO: `/v3/projects/2375e01650824224b673bea3d06af9eb` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:33.598274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2375e01650824224b673bea3d06af9eb)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:33.600420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:33.654246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:33.654811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:33.654811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:33.703015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] MFA Rules not processed for user `ce09c776cd77416091ea214b858afacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:33.704998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:33.784012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2375e01650824224b673bea3d06af9eb', 'request_id': 'req-64bb555c-072a-4c4a-9439-4c871635e07f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:33.795219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2375e01650824224b673bea3d06af9eb', 'request_id': 'req-64bb555c-072a-4c4a-9439-4c871635e07f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:33.801345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2375e01650824224b673bea3d06af9eb', 'request_id': 'req-64bb555c-072a-4c4a-9439-4c871635e07f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:33.880732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 920/1813] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:33.885168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:33.885858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:33.908815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] The token cache is being invalidate because project 2375e01650824224b673bea3d06af9eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:33.909042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:33.910044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64bb555c-072a-4c4a-9439-4c871635e07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:33.912406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 894/1814] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:33 2022] DELETE /identity/v3/projects/2375e01650824224b673bea3d06af9eb => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:33.914186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a991139-bb36-458c-829b-847b5e3010ca None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:33.922702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57424b-8d50-4c3f-947c-2413c61b514e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:33.923176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57424b-8d50-4c3f-947c-2413c61b514e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:33.923393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57424b-8d50-4c3f-947c-2413c61b514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:33.949364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-7f57424b-8d50-4c3f-947c-2413c61b514e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:33.950601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7f57424b-8d50-4c3f-947c-2413c61b514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:33.962182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:33.997342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:33.998189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:33.998466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:33.998734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:33.999444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.001359 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.019218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 895/1815] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:34.027653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.027934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.028247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.062008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:34.063522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:34.073618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 896/1816] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:34.078008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.090806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9a749-a94f-45f0-aab7-fd5a80575145 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.096042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.096908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.097333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.097772 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.098565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:34.099287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.101384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.118081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 897/1817] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:34.119283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.122856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 921/1818] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:33 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:34.127662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.130391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d1bd1-c080-43a8-98d8-28fb30e570a1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.132183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:34.206355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:34.226402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.233058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.233058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.233058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.233058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.233058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.245755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bb7b3-4d3b-4141-8fca-425d995baf1b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.251450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.252094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.253040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.253122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.255595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.259550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.361222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 898/1819] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:34 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:34.362202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.364749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:34.374571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 922/1820] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:34 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:34.374571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.386562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a3c24-7389-4c42-9889-31b9a84cda57 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.436700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68dd8-aaf4-436d-abdb-36543247b0cb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.440679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.441308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.441617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.441898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.443718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:34.454262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.454950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.455305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.455508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.456137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.458215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.508460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.510079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.519893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 899/1821] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:34.520797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.531916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0cfe8-44c6-4008-a613-6f50c2fb55c9 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.537020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.538167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.538451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.538727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.539410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.541028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.543743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 923/1822] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:34 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:34.548117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.555670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 900/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:34.559873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.564653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d56f4-7c31-4b47-ab48-649074761c09 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.573441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b8064-1bc4-49cc-a2fd-5181847f3014 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.576073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.576735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.576967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.577304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.577972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.579783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.581290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.582025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.582312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.582584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] PATH_INFO: `/v3/projects/996248694ec54255a81168f3b06341cd/users/aea36f8f711b499dab1e28931d87a07a/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.595517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 924/1824] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:34.600540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.604992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=996248694ec54255a81168f3b06341cd, user_id=aea36f8f711b499dab1e28931d87a07a, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.607883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.614669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14886c0-e3da-4b5c-b7a0-4576c866ed18 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.622320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.623088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.623361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.623633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.624762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.626204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.640238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 925/1825] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:34.644414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.657053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6d20f-1d35-40c3-b9af-97baaf81d49e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.661991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.662701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.663017 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.663308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] PATH_INFO: `/v3/projects/b42d8f109ad54098ae57caf99eb97c32/users/94446b72443042c7be0f0753172110aa/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.681426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 901/1826] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:34 2022] PUT /identity/v3/projects/996248694ec54255a81168f3b06341cd/users/aea36f8f711b499dab1e28931d87a07a/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:34.682225 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] RBAC: Authorizing `identity:create_grant(project_id=b42d8f109ad54098ae57caf99eb97c32, user_id=94446b72443042c7be0f0753172110aa, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.685140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f6916-6fa2-47ed-830b-6f62282182db admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.685483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.764596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35d46c-ef50-4234-bedf-c0a0cb80f0cf admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.769782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.770416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.770699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.773315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.773315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.774852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.785823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 902/1827] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:34.789462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:34.796457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 926/1828] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:34 2022] PUT /identity/v3/projects/b42d8f109ad54098ae57caf99eb97c32/users/94446b72443042c7be0f0753172110aa/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:34.808003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978bea1-efc0-43b8-af61-e4af23360ac0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:34.813054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.813360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.813728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.818272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:34.878475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] MFA Rules not processed for user `94446b72443042c7be0f0753172110aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:34.882286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:34.912990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:34.916508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:34.916508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:34.916508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] PATH_INFO: `/v3/projects/996248694ec54255a81168f3b06341cd/users/aea36f8f711b499dab1e28931d87a07a/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:34.918600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=996248694ec54255a81168f3b06341cd, user_id=aea36f8f711b499dab1e28931d87a07a, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:34.922418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:34.951994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 903/1829] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:34 2022] PUT /identity/v3/projects/996248694ec54255a81168f3b06341cd/users/aea36f8f711b499dab1e28931d87a07a/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:35.001600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:35.008576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 927/1830] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:35.013132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.097932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e29591-1dbd-44e9-8fec-aba88fc1ab61 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.102791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.103522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.103875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.104155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.104789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.106718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.231673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 904/1831] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:35 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:35.233397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.246048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11d00c-1e3d-4743-aeba-822317c0da9b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.304397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.305161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.305440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.305708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.306344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.308322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.404851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:35.409897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 928/1832] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:35 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:35.414089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.427753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b389874-c61c-4f96-af07-2a453263ec65 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.433112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.433750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.434030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.434299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.434995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.436642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.447313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 905/1833] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:35.451750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.462603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8574e-4a08-4a86-8592-9c6a0e942355 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.467378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.468086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.468385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.468664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] PATH_INFO: `/v3/projects/c7101209b25a4603b3f37a287ed3c440/users/20e7afc30a4b4d77a5af2cb5e3a3a811/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.485714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7101209b25a4603b3f37a287ed3c440, user_id=20e7afc30a4b4d77a5af2cb5e3a3a811, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.488644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.592690 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 929/1834] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:35 2022] PUT /identity/v3/projects/c7101209b25a4603b3f37a287ed3c440/users/20e7afc30a4b4d77a5af2cb5e3a3a811/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:35.596874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.607434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1965981-207f-4f2f-a8ad-17af283478da admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.676414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.677441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.677541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.677786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.678513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.683082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115a455-03ff-4925-9368-dc4246f78ab7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.695306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 906/1835] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:35.696552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:35.711682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3ce98-056a-484c-8803-6de83fca4d5e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:35.718675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:35.719858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.720394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.727633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] PATH_INFO: `/v3/projects/c7101209b25a4603b3f37a287ed3c440/users/20e7afc30a4b4d77a5af2cb5e3a3a811/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.730190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7101209b25a4603b3f37a287ed3c440, user_id=20e7afc30a4b4d77a5af2cb5e3a3a811, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:35.733078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:35.750251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 930/1836] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:35 2022] PUT /identity/v3/projects/c7101209b25a4603b3f37a287ed3c440/users/20e7afc30a4b4d77a5af2cb5e3a3a811/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:35.762104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:35.762627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:35.763541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:35.815269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] MFA Rules not processed for user `20e7afc30a4b4d77a5af2cb5e3a3a811`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:35.817012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:35.917436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 907/1837] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:35.983755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:36.049830 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4afa7-ac42-44f9-8a83-5c8ecbd3481e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:36.053764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:36.054401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:36.054758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:36.055084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:36.065721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:36.068041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:36.080010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 931/1838] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:36.361659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:36.378619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899e74f-71b6-4bfb-8718-64eb6bedb19c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:36.384595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:36.385590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:36.386228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:36.386974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:36.389716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:36.451493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:36.517194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:36.518855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:36.535021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 908/1839] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:37.876509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:37.900664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ff31e-a476-44b1-8752-121981bd3ea0 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:37.939587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e7afc30a4b4d77a5af2cb5e3a3a811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7101209b25a4603b3f37a287ed3c440', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:37.940447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:37.940570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:37.940764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:37.942509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:37.942509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:37.945020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:37.968422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 932/1840] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:37.972494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:37.985532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e6205-3e91-4770-9c4f-3445c9ff0f73 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:37.996508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e7afc30a4b4d77a5af2cb5e3a3a811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7101209b25a4603b3f37a287ed3c440', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:37.997255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:37.997598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:37.998101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:37.998504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:38.000527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:38.134774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 909/1841] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:37 2022] POST /identity/v3/projects => generated 353 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:38.139289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:38.149602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d41e1-9665-4280-acad-ce85b6770b73 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:38.211143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:38.238737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e7afc30a4b4d77a5af2cb5e3a3a811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7101209b25a4603b3f37a287ed3c440', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:38.239609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:38.240060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:38.240511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:38.241875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:38.242348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:38.244187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:38.264931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 933/1842] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:38.330544 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:38.344303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd3b54-bf73-4d61-864c-aed0a6286147 tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:38.350040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20e7afc30a4b4d77a5af2cb5e3a3a811', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7101209b25a4603b3f37a287ed3c440', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:38.351489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:38.355506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:38.355506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] PATH_INFO: `/v3/projects/aedfe853a8d945438365c37beb9c408e` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:38.355506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aedfe853a8d945438365c37beb9c408e)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:38.355506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:38.506488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aedfe853a8d945438365c37beb9c408e', 'request_id': 'req-6d41c41c-2bad-4027-a285-4f7c2da11adb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:38.508290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aedfe853a8d945438365c37beb9c408e', 'request_id': 'req-6d41c41c-2bad-4027-a285-4f7c2da11adb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:38.515641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aedfe853a8d945438365c37beb9c408e', 'request_id': 'req-6d41c41c-2bad-4027-a285-4f7c2da11adb'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:38.592866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] The token cache is being invalidate because project aedfe853a8d945438365c37beb9c408e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:38.593303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:38.593949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:38.596148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 910/1843] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:38 2022] DELETE /identity/v3/projects/aedfe853a8d945438365c37beb9c408e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:39.483115 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:39.486179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:39.601907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471eb087-ac97-4d7a-a893-9af16acae66d tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:39.688210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:39.690101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:39.690393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:39.691140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] PATH_INFO: `/v3/users/94446b72443042c7be0f0753172110aa` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:39.693218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94446b72443042c7be0f0753172110aa)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:39.709281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:40.002407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94446b72443042c7be0f0753172110aa'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.039185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94446b72443042c7be0f0753172110aa'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.040509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94446b72443042c7be0f0753172110aa'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.046799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:40.051429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:40.064824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41c41c-2bad-4027-a285-4f7c2da11adb tempest-VolumeQuotaClassesTest-1491232214 tempest-VolumeQuotaClassesTest-1491232214-project-admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:40.073924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:40.075839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:40.075940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:40.076185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] PATH_INFO: `/v3/users/06cbac6e08094f2a8667f74799352868` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:40.078172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cbac6e08094f2a8667f74799352868)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:40.081122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:40.112656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94446b72443042c7be0f0753172110aa'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.113575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-0e4c9a94-e58e-4722-b0b2-86d8090248b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94446b72443042c7be0f0753172110aa'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.136490 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:40.145667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 934/1844] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:39 2022] DELETE /identity/v3/users/94446b72443042c7be0f0753172110aa => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:40.150923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:40.151229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:40.151488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:40.180181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:40.181691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:40.269616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 935/1845] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:40.279636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:40.281288 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:40.316134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cbac6e08094f2a8667f74799352868'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.325859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cbac6e08094f2a8667f74799352868'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.327059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cbac6e08094f2a8667f74799352868'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.366425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcba08-8bb4-447b-a7ad-f78e0c8762b0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:40.371569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:40.372638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:40.372923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:40.373200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:40.385373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:40.386844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cb715-48e8-4793-abe0-a7d2b94083f4 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:40.399143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cbac6e08094f2a8667f74799352868'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.400058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cbac6e08094f2a8667f74799352868'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:40.406905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 936/1846] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:40.416079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:40.424988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 911/1847] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:40 2022] DELETE /identity/v3/users/06cbac6e08094f2a8667f74799352868 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:40.429776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:40.431131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:40.431491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:40.457395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:40.458641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:40.571456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 937/1848] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:40.587514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:40.590724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:40.713388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d99aee-7969-4cac-af60-622e975bf81c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:40.726645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:40.727907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:40.730180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:40.730670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:40.745294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:40.746765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:40.761693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 912/1849] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:41.032736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:41.034336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:41.048317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b011bc8-9808-4f21-9ec7-95cf262444ae None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:41.053876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:41.055300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:41.055579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:41.055849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] PATH_INFO: `/v3/projects/b42d8f109ad54098ae57caf99eb97c32` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:41.057009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b42d8f109ad54098ae57caf99eb97c32)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:41.059042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:41.173955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:41.181921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b42d8f109ad54098ae57caf99eb97c32', 'request_id': 'req-d30784ef-2557-4530-b87f-17238dd08e39'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.188788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b42d8f109ad54098ae57caf99eb97c32', 'request_id': 'req-d30784ef-2557-4530-b87f-17238dd08e39'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.194595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b42d8f109ad54098ae57caf99eb97c32', 'request_id': 'req-d30784ef-2557-4530-b87f-17238dd08e39'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.320570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] The token cache is being invalidate because project b42d8f109ad54098ae57caf99eb97c32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:41.320902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.321601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.324482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 938/1850] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:41 2022] DELETE /identity/v3/projects/b42d8f109ad54098ae57caf99eb97c32 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:41.328393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:41.344203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b3f40-9b6e-4b90-b4f0-8b00552d037d service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:41.358081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:41.359955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:41.447063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:41.447812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:41.448330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:41.448673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] PATH_INFO: `/v3/users/20e7afc30a4b4d77a5af2cb5e3a3a811` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:41.451444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] RBAC: Authorizing `identity:delete_user(user_id=20e7afc30a4b4d77a5af2cb5e3a3a811)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:41.454291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:41.480344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30784ef-2557-4530-b87f-17238dd08e39 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:41.498422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:41.499122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:41.499595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:41.500027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] PATH_INFO: `/v3/projects/fcc46bf86cf044be81715a4c51d58c62` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:41.501674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc46bf86cf044be81715a4c51d58c62)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:41.505444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:41.718254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc46bf86cf044be81715a4c51d58c62', 'request_id': 'req-80b6411d-212f-421e-a965-92fd5280e64e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.718937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20e7afc30a4b4d77a5af2cb5e3a3a811'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.727028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc46bf86cf044be81715a4c51d58c62', 'request_id': 'req-80b6411d-212f-421e-a965-92fd5280e64e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.728778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20e7afc30a4b4d77a5af2cb5e3a3a811'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.729889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20e7afc30a4b4d77a5af2cb5e3a3a811'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.735135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc46bf86cf044be81715a4c51d58c62', 'request_id': 'req-80b6411d-212f-421e-a965-92fd5280e64e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.817904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20e7afc30a4b4d77a5af2cb5e3a3a811'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.818554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d90e14f3-d4cf-4619-b3bd-c6cafbd0412a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20e7afc30a4b4d77a5af2cb5e3a3a811'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.838229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 913/1851] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:41 2022] DELETE /identity/v3/users/20e7afc30a4b4d77a5af2cb5e3a3a811 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:41.839641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:41.839927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:41.840206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:41.874551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] MFA Rules not processed for user `aea36f8f711b499dab1e28931d87a07a`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:41.875939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:41.925424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] The token cache is being invalidate because project fcc46bf86cf044be81715a4c51d58c62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:41.925965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.927012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-80b6411d-212f-421e-a965-92fd5280e64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:41.936761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 939/1852] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:41 2022] DELETE /identity/v3/projects/fcc46bf86cf044be81715a4c51d58c62 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:41.938273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73cca-db2c-4fcd-add5-9154793028ab None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:41.938564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73cca-db2c-4fcd-add5-9154793028ab None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:41.938854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf73cca-db2c-4fcd-add5-9154793028ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:41.966098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-bcf73cca-db2c-4fcd-add5-9154793028ab None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:41.967606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bcf73cca-db2c-4fcd-add5-9154793028ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.015129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 914/1853] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:42.016376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.017935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.096061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 940/1854] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:42.106398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.106680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.106975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.135813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5d9b1-f2ec-49ef-9400-6e45b56116c9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.135921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:42.135921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.136456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.137294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.138247 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.138736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.141405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.146515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 941/1855] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:42.147642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.158217 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9b03b-b473-4a53-b50b-a542519a81f4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.163123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.164524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.164889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.165220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.165917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:42.166276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.167971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.181410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 942/1856] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:42.185607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.200761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccdd5b-cd15-4813-a6bc-49bbc37f8c9a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.219752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.222036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.223974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.224776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.225059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.225329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.226012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.228487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.235931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 915/1857] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:42.253110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:42.373950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 943/1858] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:42.379215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.390141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3877f53-f627-4f88-9027-4c10a6bee0b6 service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.407871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.409963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.517070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.517070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.517070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.517070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.517422 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.518373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.544752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b174c7e-1e18-454e-b826-abd4ae09484f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.551170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.563139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.563627 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.564012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] PATH_INFO: `/v3/projects/c7101209b25a4603b3f37a287ed3c440` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.565541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7101209b25a4603b3f37a287ed3c440)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.568786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.611957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:42.619502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 916/1859] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:42.623246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.641610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953e914-2690-466c-8cb8-7c0981c50974 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.646895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.648813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.649149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.649589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.650656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.652558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.669123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 917/1860] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:42.670080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.720381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d238878-b809-4f5f-a670-527d7370f02b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.730333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.731576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.731861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.732133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.732791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.734398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.756660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7101209b25a4603b3f37a287ed3c440', 'request_id': 'req-bc465bf6-30ab-4458-b900-0193ddac3198'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:42.769694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7101209b25a4603b3f37a287ed3c440', 'request_id': 'req-bc465bf6-30ab-4458-b900-0193ddac3198'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:42.776166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 918/1861] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:42.784071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7101209b25a4603b3f37a287ed3c440', 'request_id': 'req-bc465bf6-30ab-4458-b900-0193ddac3198'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:42.787008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:42.800315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ac6b-6381-461b-a2f6-f07befc66338 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:42.870579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:42.871075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.871368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.871629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] PATH_INFO: `/v3/projects/fcb757b16b4048c69cdbb2272dfed9c3/users/33a05e874a294177943950eee03ec35e/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.888126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb757b16b4048c69cdbb2272dfed9c3, user_id=33a05e874a294177943950eee03ec35e, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:42.892135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f124ce-a2a7-42ca-8db0-9f12b24a6792 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:42.920208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] The token cache is being invalidate because project c7101209b25a4603b3f37a287ed3c440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:42.920507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:42.921369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bc465bf6-30ab-4458-b900-0193ddac3198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:42.922761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 944/1862] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:42 2022] DELETE /identity/v3/projects/c7101209b25a4603b3f37a287ed3c440 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:42.927783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676bc3d-9698-4771-b83f-2de1d7c01357 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.928524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676bc3d-9698-4771-b83f-2de1d7c01357 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.928875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676bc3d-9698-4771-b83f-2de1d7c01357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.951770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1676bc3d-9698-4771-b83f-2de1d7c01357 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:42.952986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1676bc3d-9698-4771-b83f-2de1d7c01357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:42.961285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 945/1863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:42.969176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.969465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.969744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:42.972974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 919/1864] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:42 2022] PUT /identity/v3/projects/fcb757b16b4048c69cdbb2272dfed9c3/users/33a05e874a294177943950eee03ec35e/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:42.988868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:42.989158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:42.989434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.002373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:43.003997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.035963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] MFA Rules not processed for user `33a05e874a294177943950eee03ec35e`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:43.037728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.087781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 946/1865] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:43.092016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.102180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae8b55f-5d78-4aed-a8fb-68dbd34ac038 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.108208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.108886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.109170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.109442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.110089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:43.110687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.112458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.122126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 920/1866] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:43.126935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.129706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.131153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 947/1867] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:19:43.136602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.150696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a934c-d003-4e46-bc59-818114cb7673 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.155798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd87d7-ece4-4a2e-b1d6-cddbc743fe7a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.158261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.160092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.160304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.160611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.161257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.161837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.162790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.163204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.163788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.164176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.165335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.168870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.258444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 948/1868] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:43.262286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.283923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 921/1869] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:43.292220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.332318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcec191-8eaa-466f-907d-89845b91f434 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.396362 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f39caa-1038-4d59-945d-efd7685e3363 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.403320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.404166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.404677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.404799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.405278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.405896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.406182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.406467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.406579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.407539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.409197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.410216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.525283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 922/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:43.528641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 949/1871] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:43.529414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.530039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.539943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7670d6-2c59-47f8-81e5-75c9568e004a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.540516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44057fd1-f73b-4a3c-a8c7-af1a1a3e5025 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.545635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.546240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.546344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.546516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.546791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.547104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.547387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.547488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.547667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.548316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.549120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.551482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.561303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 950/1872] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:43.563417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 923/1873] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:43.567242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.567402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.577828 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748711c3-9302-4105-a2bc-dbe17ee53a2e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.579980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ae8b3-a78b-4213-88ef-d2698c810933 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.583600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.584492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.588186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.588186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.588186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.588532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.588625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.588699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.588699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.588699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.588699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.591654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.605103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 951/1874] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:43.606384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.607626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 924/1875] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:43.611719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.620853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37997613-f6c4-4c55-a0a8-9b5fd34a7008 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.622291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0ebaa-7b7a-4624-b2d3-dade6bfbedde admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.625494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.626146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.626421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.626691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] PATH_INFO: `/v3/projects/97d4eb46e41c46629c55105184549652/users/53921edda4a34513a676b205668d4f5e/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.627224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.628212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.628496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.628820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] PATH_INFO: `/v3/projects/8576eb9289f147f2ab9ba6f1c00b3ffd/users/a79972df53cf4245be9c45a9b579cff9/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.645875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d4eb46e41c46629c55105184549652, user_id=53921edda4a34513a676b205668d4f5e, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.646963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8576eb9289f147f2ab9ba6f1c00b3ffd, user_id=a79972df53cf4245be9c45a9b579cff9, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.649360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea65e4-88d1-4489-a56c-c00e34f64a82 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.650143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411655b1-863e-488e-a359-8469389b0454 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.761574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 925/1876] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:43 2022] PUT /identity/v3/projects/8576eb9289f147f2ab9ba6f1c00b3ffd/users/a79972df53cf4245be9c45a9b579cff9/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:43.766363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 952/1877] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:43 2022] PUT /identity/v3/projects/97d4eb46e41c46629c55105184549652/users/53921edda4a34513a676b205668d4f5e/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:43.766835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.766835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.767077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.777806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.777806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.777957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.791093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:43.792868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.812772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] MFA Rules not processed for user `53921edda4a34513a676b205668d4f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:43.814994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.874705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 926/1878] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:43.884700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.886114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:43.907202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 953/1879] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:43.911613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:43.924901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37b51b-a72e-4e4f-821a-1595351b3422 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.932045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.933641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.933991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.934348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.935106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.937252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:43.973460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d069ec-b2b3-4bf7-bb14-af44fdd284b6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:43.981620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:43.983401 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:43.983871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:43.984710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:43.994583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:43.996058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.006699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 927/1880] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:44.007282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.008770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.022926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b056f0e-5397-4fa5-993e-90a2eb025a00 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.036427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.037207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.037403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.037707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] PATH_INFO: `/v3/users/30974466e60244b5993d5d79e4a48f3a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.039828 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30974466e60244b5993d5d79e4a48f3a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.041703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.059455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 954/1881] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:43 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:44.065184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.142273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926ab33-5715-48fa-8acd-94dc96b4be53 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.146942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.147675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.147956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.149258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.149496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.151445 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.255195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 955/1882] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:44 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:44.256262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.266254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0f416-b005-4b70-bacd-e6c289a29e0e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.270774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.271391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.271668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.271937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.272599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.274765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.284418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 956/1883] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:44.288489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.298330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737878-556e-4066-a4c6-8c5fbb02bb2c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.304736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.305605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.306207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.306636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] PATH_INFO: `/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa/users/8582c34067934b59b71cfd6befb7fee9/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.315166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30974466e60244b5993d5d79e4a48f3a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:44.324984 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30974466e60244b5993d5d79e4a48f3a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:44.325329 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1ce6b3c8df4f5999b8759f311abbfa, user_id=8582c34067934b59b71cfd6befb7fee9, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.325854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30974466e60244b5993d5d79e4a48f3a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:44.329282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.382468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30974466e60244b5993d5d79e4a48f3a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:44.383766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-0a729779-a2bd-498f-8b2d-308bdf7d28b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30974466e60244b5993d5d79e4a48f3a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:44.390063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 957/1884] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:44 2022] PUT /identity/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa/users/8582c34067934b59b71cfd6befb7fee9/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:44.394977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.411183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.419995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 928/1885] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:44 2022] DELETE /identity/v3/users/30974466e60244b5993d5d79e4a48f3a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:44.424843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.430198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.430198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.450642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:44.451909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.500064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3124c2b-d3b8-4cd5-8128-ee4a43bb26ef admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.535070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 929/1886] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:44.556419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.558043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.571393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.572156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.572443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.572715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.573413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.575071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.589446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 958/1887] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:44.590945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.601238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfc7e9-056a-4770-8907-dbf3dac6baff admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.607095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:44.611784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.611784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.611784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.611784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] PATH_INFO: `/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa/users/8582c34067934b59b71cfd6befb7fee9/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.614609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1ce6b3c8df4f5999b8759f311abbfa, user_id=8582c34067934b59b71cfd6befb7fee9, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.618701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f1609-9d67-477a-b67a-9c38e4c5cab1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.634967 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 959/1888] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:44 2022] PUT /identity/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa/users/8582c34067934b59b71cfd6befb7fee9/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:44.647631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.650551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.650551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.661702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5d5f1-995e-4748-a308-049895a67ec5 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.681577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] MFA Rules not processed for user `8582c34067934b59b71cfd6befb7fee9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:44.684492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.727560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.728387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.728674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.728949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.777891 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 960/1889] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:44.790152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:44.791573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.805817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.807283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.817324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 930/1890] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:44.826185 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:44.857336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997701f5-2546-4fc8-9d53-d41652efbd5a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:44.861094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:44.861708 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:44.862070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:44.862262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:44.864323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:44.947829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:44.950915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cab3c-08fd-4c58-9481-9ceacb4320e9 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:44.962576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 961/1891] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:45.057430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:45.472472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:45.474508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:45.494199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b382c-6e4c-4b7f-a1e7-6f314b639f56 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:45.499384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:45.500050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:45.500326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:45.500593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] PATH_INFO: `/v3/projects/df5f915d57114c8a8a4e376efbe3094c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:45.501718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5f915d57114c8a8a4e376efbe3094c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:45.504291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:45.556271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:45.556557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:45.556833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:45.596788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] MFA Rules not processed for user `a79972df53cf4245be9c45a9b579cff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:45.606009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:45.645848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5f915d57114c8a8a4e376efbe3094c', 'request_id': 'req-01b74790-3ddd-4539-b185-698dec4df5b0'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:45.653085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5f915d57114c8a8a4e376efbe3094c', 'request_id': 'req-01b74790-3ddd-4539-b185-698dec4df5b0'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:45.659280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5f915d57114c8a8a4e376efbe3094c', 'request_id': 'req-01b74790-3ddd-4539-b185-698dec4df5b0'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:45.714900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 962/1892] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:45.724092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:45.725189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:45.738065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9917a-e2d0-41e6-a323-b411f0487da9 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:45.792341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:45.792341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:45.792341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:45.792341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:45.819846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] The token cache is being invalidate because project df5f915d57114c8a8a4e376efbe3094c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:45.820183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:45.822464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:45.832222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 931/1893] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:45 2022] DELETE /identity/v3/projects/df5f915d57114c8a8a4e376efbe3094c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:45.832222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:45.832222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:45.887947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:45.902077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:45.903965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c34f9-af31-4d50-8d45-8181968a25b6 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:45.915166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 963/1894] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:45.952847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:45.953383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:45.953864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:45.979890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] MFA Rules not processed for user `ce09c776cd77416091ea214b858afacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:45.981569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:45.989368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b74790-3ddd-4539-b185-698dec4df5b0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:46.001246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:46.001921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:46.002205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:46.002485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] PATH_INFO: `/v3/users/e92ae010a33e4c778414f1354a4ba1f8` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:46.006438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92ae010a33e4c778414f1354a4ba1f8)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:46.007605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:46.011472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:46.097286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 964/1895] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:46.237497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92ae010a33e4c778414f1354a4ba1f8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:46.244375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92ae010a33e4c778414f1354a4ba1f8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:46.245048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92ae010a33e4c778414f1354a4ba1f8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:46.309662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92ae010a33e4c778414f1354a4ba1f8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:46.310160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92ae010a33e4c778414f1354a4ba1f8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:46.333478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 932/1896] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:45 2022] DELETE /identity/v3/users/e92ae010a33e4c778414f1354a4ba1f8 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:46.775276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:46.789052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72147ab-6533-4a32-8a8b-995964ddfd15 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:46.855382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:46.856087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:46.856366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:46.856677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] PATH_INFO: `/v3/projects/808b9a0b27e247868c3988871137e55e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:46.862664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] RBAC: Authorizing `identity:delete_project(project_id=808b9a0b27e247868c3988871137e55e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:46.862664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:47.009684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808b9a0b27e247868c3988871137e55e', 'request_id': 'req-8dfd241d-0fc6-457f-993c-1b62df93acde'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.023347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808b9a0b27e247868c3988871137e55e', 'request_id': 'req-8dfd241d-0fc6-457f-993c-1b62df93acde'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.026501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808b9a0b27e247868c3988871137e55e', 'request_id': 'req-8dfd241d-0fc6-457f-993c-1b62df93acde'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.121675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] The token cache is being invalidate because project 808b9a0b27e247868c3988871137e55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:47.130571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.136018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.141911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 965/1897] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:46 2022] DELETE /identity/v3/projects/808b9a0b27e247868c3988871137e55e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:47.142559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:47.144533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:47.159110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7f51-f14f-47b4-9b8e-f740ce1e91f0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:47.220958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:47.221600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:47.221878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:47.222185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] PATH_INFO: `/v3/users/124fb50cf80742b791d1cdb644a0672d` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:47.223376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124fb50cf80742b791d1cdb644a0672d)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:47.225323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:47.368105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:47.444456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124fb50cf80742b791d1cdb644a0672d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.453178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124fb50cf80742b791d1cdb644a0672d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.454173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124fb50cf80742b791d1cdb644a0672d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.498996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:47.521507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124fb50cf80742b791d1cdb644a0672d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.522225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124fb50cf80742b791d1cdb644a0672d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:47.541092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 933/1898] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:47 2022] DELETE /identity/v3/users/124fb50cf80742b791d1cdb644a0672d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:48.089018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.090637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.108640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd241d-0fc6-457f-993c-1b62df93acde admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.171890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.172421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.172705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.172980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] PATH_INFO: `/v3/projects/7747cb564ae44ddf8072d0fc81147938` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.174151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7747cb564ae44ddf8072d0fc81147938)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.176087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.326120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7747cb564ae44ddf8072d0fc81147938', 'request_id': 'req-05bda48f-26bb-42b9-af55-000a746722bc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:48.332566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7747cb564ae44ddf8072d0fc81147938', 'request_id': 'req-05bda48f-26bb-42b9-af55-000a746722bc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:48.338650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7747cb564ae44ddf8072d0fc81147938', 'request_id': 'req-05bda48f-26bb-42b9-af55-000a746722bc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:48.360913 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.362406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.421146 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968b12b-4780-4e6f-ae9c-1a63d33a13b7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.422343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] The token cache is being invalidate because project 7747cb564ae44ddf8072d0fc81147938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:48.422650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:48.423290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-05bda48f-26bb-42b9-af55-000a746722bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:48.424839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 966/1899] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:48 2022] DELETE /identity/v3/projects/7747cb564ae44ddf8072d0fc81147938 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:48.432030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e156a8-1540-4242-a1bc-1d56bb436c61 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.432333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e156a8-1540-4242-a1bc-1d56bb436c61 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.432608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e156a8-1540-4242-a1bc-1d56bb436c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.451985 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-81e156a8-1540-4242-a1bc-1d56bb436c61 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:48.453190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-81e156a8-1540-4242-a1bc-1d56bb436c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.475473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.476361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.476666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.476923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.479340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.519971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 967/1900] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:48.529271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.529753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.530024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.540796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.542278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.551360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 934/1901] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:48.551976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:48.553536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.563229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 968/1902] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:48.567339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.579783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b8c78-0f8d-416c-bf44-d3868c463bb1 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.584523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.585290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.585575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.585856 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.587102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:48.587417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.589125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.600753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 935/1903] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:48.605754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.616455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202a58e-85d5-497e-8aa2-95ec0c1deb60 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.628022 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.628645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.628923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.629193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.629973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.632888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc946c80-1214-4314-94dd-6acb82a7ef40 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.651585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:48.752878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 969/1904] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:48 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:48.758089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.773581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a0e6-bd93-456c-86ee-e51c350259d5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.803542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.804309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.805355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.831843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] MFA Rules not processed for user `a79972df53cf4245be9c45a9b579cff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:48.833394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:48.851455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.852210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.852494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.852771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.853405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.855599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.893987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 970/1905] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:48.934620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 936/1906] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:48 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:48.935179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.946979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca729dd-a473-41e0-a066-5650177c3f69 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.952160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.952887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.953121 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.953420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.954070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.955788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:48.969309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 971/1907] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:48.973742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:48.984059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ceb560-1a43-4b5d-a060-9429e2f5807e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:48.988865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:48.989469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:48.989746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:48.990017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:48.990648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:48.992371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555519c-98bf-4362-90d1-51c31c4a7db4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:49.004032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 937/1908] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:49.009327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:49.019852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f893c-69ec-42a9-a37f-0dc4b897e82c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:49.025385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:49.026635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:49.026635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:49.026635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] PATH_INFO: `/v3/projects/e378bfab9d91492e81320139fed867c6/users/1f737b4d0531425cbc803fcd15a3ea1f/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:49.042386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e378bfab9d91492e81320139fed867c6, user_id=1f737b4d0531425cbc803fcd15a3ea1f, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:49.045372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:49.143617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 972/1909] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:49 2022] PUT /identity/v3/projects/e378bfab9d91492e81320139fed867c6/users/1f737b4d0531425cbc803fcd15a3ea1f/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:49.148761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:49.149083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:49.149371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:49.169023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:49.170230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:49.182125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:49.236713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 938/1910] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:49.247247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:49.249154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:49.320184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba72fd2-d552-43c6-88ed-9b71a1eee351 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:49.325241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:49.326196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:49.326566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:49.326864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:49.338842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:49.340299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:49.352721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 973/1911] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:49.871956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:50.270553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:50.272711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:50.348013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f8d02-744f-4db1-9e1d-a7368a3dd0af None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:50.372065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:50.372709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:50.373024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:50.373302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:50.376064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:50.442234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:50.458659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:50.460133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c00b24-3a46-4b93-bfbf-2adb7b8bdd2b service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:50.470563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 939/1912] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:50.939863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:52.619200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:53.266364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:55.503145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:55.505743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:55.542638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc01a1-25b3-4752-b8eb-a3063a72d404 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:55.554073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:55.562672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:55.562672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:55.562672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] PATH_INFO: `/v3/users/0b22256920ae421f9af326d68d412375` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:55.579049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b22256920ae421f9af326d68d412375)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:55.584938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:55.736148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:55.736570 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:55.736864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:55.779881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:55.799195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] MFA Rules not processed for user `1f737b4d0531425cbc803fcd15a3ea1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:55.799195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:55.917808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b22256920ae421f9af326d68d412375'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:55.925919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 940/1913] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:55.930355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:55.934152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b22256920ae421f9af326d68d412375'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:55.934968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b22256920ae421f9af326d68d412375'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:55.945265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b127bf-cdbc-4a01-b7bb-69ed2452568e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:55.950078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:55.950751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:55.951129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:55.951410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:55.952186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:55.954054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:55.991634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b22256920ae421f9af326d68d412375'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:55.992415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8267f97-6294-45ad-891a-c53d04e8ea16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b22256920ae421f9af326d68d412375'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:56.011642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:56.022726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 974/1914] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:55 2022] DELETE /identity/v3/users/0b22256920ae421f9af326d68d412375 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:56.027905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.028212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.028949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.059815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 941/1915] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:55 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:56.062747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.065211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:56.066684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:56.067620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:56.155821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c2ad5-ae3d-4420-bcd5-0e1beafffaff admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.162029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.163326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.164162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.164969 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.166180 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.169606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.182875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 975/1916] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:56.190855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.193347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:56.256647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e3a1f-44f1-4175-ada3-b720ef773254 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.261650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.262493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.263102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.263204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.271870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 942/1917] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:56 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:56.276003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.285072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.286576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.297731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aaed6-a461-4d64-a912-6195e3650604 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.297938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 976/1918] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:56.304431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.305398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.306051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.306051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.306792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.308465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.322462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 943/1919] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:56.324070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.335358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c78369-7bf2-4fc1-bea0-14beaeb8cd5f service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.341042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.345243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.345243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.345243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] PATH_INFO: `/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01/users/28e0de7076904034a98885c8ebeddce3/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.359386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed68e985764bc5b139aa5a9de6bd01, user_id=28e0de7076904034a98885c8ebeddce3, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.362091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.438392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 977/1920] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:56 2022] PUT /identity/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01/users/28e0de7076904034a98885c8ebeddce3/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:56.442365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.452846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065dbabd-0f30-417e-b30b-e283bb4d2075 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.513427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.514235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.514523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.514800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.515506 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.517117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.527604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 944/1921] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:56.532201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.542998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dafaf-9177-43f3-994c-81ca8c032e68 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.559955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.560661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.561172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.561625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] PATH_INFO: `/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01/users/28e0de7076904034a98885c8ebeddce3/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.564228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ed68e985764bc5b139aa5a9de6bd01, user_id=28e0de7076904034a98885c8ebeddce3, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.567391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.580682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 978/1922] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:56 2022] PUT /identity/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01/users/28e0de7076904034a98885c8ebeddce3/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:56.624494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:56.815007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.816980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:56.835356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eea051-a116-4380-bbb0-15e731730a72 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:56.980796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:56.982620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:56.987710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:56.992986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:56.992986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:56.992986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] PATH_INFO: `/v3/projects/924211a2eedf4c00ad49f2bcdf1c1431` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:56.992986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924211a2eedf4c00ad49f2bcdf1c1431)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:56.992986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:56.998095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c36b7-f167-4056-b541-08d9eccb1ad1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.015106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.015766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.016045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.016382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] PATH_INFO: `/v3/users/33a05e874a294177943950eee03ec35e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.017889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a05e874a294177943950eee03ec35e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.020868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.185621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924211a2eedf4c00ad49f2bcdf1c1431', 'request_id': 'req-d96d824a-2fc5-48e6-8bf5-8658942a9d75'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.192879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924211a2eedf4c00ad49f2bcdf1c1431', 'request_id': 'req-d96d824a-2fc5-48e6-8bf5-8658942a9d75'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.202953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924211a2eedf4c00ad49f2bcdf1c1431', 'request_id': 'req-d96d824a-2fc5-48e6-8bf5-8658942a9d75'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.250098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a05e874a294177943950eee03ec35e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.257751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a05e874a294177943950eee03ec35e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.259130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a05e874a294177943950eee03ec35e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.292621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] The token cache is being invalidate because project 924211a2eedf4c00ad49f2bcdf1c1431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:57.292998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.293765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d96d824a-2fc5-48e6-8bf5-8658942a9d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.299234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 945/1923] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:56 2022] DELETE /identity/v3/projects/924211a2eedf4c00ad49f2bcdf1c1431 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:57.302148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21c2a-f76a-4a8e-87dc-eab2f50f0da9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.302526 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21c2a-f76a-4a8e-87dc-eab2f50f0da9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.302923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd21c2a-f76a-4a8e-87dc-eab2f50f0da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.327513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5fd21c2a-f76a-4a8e-87dc-eab2f50f0da9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:57.328934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5fd21c2a-f76a-4a8e-87dc-eab2f50f0da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:57.363817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a05e874a294177943950eee03ec35e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.368028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f3b90f41-09c6-4368-82d8-9fdef1f44288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a05e874a294177943950eee03ec35e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:57.398203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 979/1924] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:56 2022] DELETE /identity/v3/users/33a05e874a294177943950eee03ec35e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:19:57.399996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.401188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.401670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.436533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:57.436801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:57.486954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 946/1925] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:57.494268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.494627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.495008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.512269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 980/1926] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:57.518760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:57.520188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:57.523903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:57.526165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:57.534601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 947/1927] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:19:57.535947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:57.538934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:57.546922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1e1e7-4161-49b3-af61-7366429c958d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.553225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.553844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.554124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.554580 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.555709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:19:57.556104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.557886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.574622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 948/1928] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:19:57.579554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:57.592141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3bb08-d5f0-4609-ae30-c29d2c30e1e1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.598447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.599220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.599565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.599910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.600842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.603962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.619756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d4bc47-a0bf-417e-a091-9dd3e012d750 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.625401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.626165 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.626512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.627135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.638979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.640527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.652173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 981/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:19:57.701015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:57.738088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 949/1930] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:19:57 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:19:57.742884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:57.753480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1564b72-30d0-4240-bdda-663aa2941b95 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.830997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.831526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.831894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.832345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.833171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.835431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.939493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:57.957765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 982/1931] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:19:57 2022] POST /identity/v3/users => generated 284 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:19:57.962500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023dc21-c3b2-4e58-b480-6c1fb0ae98f5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:57.967762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:57.969355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:57.969645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:57.969926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:57.970579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:57.972242 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:57.986698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 950/1932] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:19:57.991127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:58.006149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155fe700-9946-42a9-b67d-7f00f8aea023 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:58.022908 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:58.023386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.023737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.024184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.026099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:58.028483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9342c-dc3d-49cc-8c90-6c56d50b5f26 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:58.047776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 983/1933] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:19:58.051167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:58.066925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b5659-9553-4266-b8e1-c49de8ca4fdc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:58.073027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:58.073840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.074173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.074501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] PATH_INFO: `/v3/projects/d34ea9c8581c4a53a949b046a7776789/users/f6f611c8544040439f53e1ad2e98ebf7/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.100154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34ea9c8581c4a53a949b046a7776789, user_id=f6f611c8544040439f53e1ad2e98ebf7, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:58.104047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:58.202254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 951/1934] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:19:58 2022] PUT /identity/v3/projects/d34ea9c8581c4a53a949b046a7776789/users/f6f611c8544040439f53e1ad2e98ebf7/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:58.213646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.214352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.214664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.251137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] MFA Rules not processed for user `f6f611c8544040439f53e1ad2e98ebf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:19:58.252411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:58.308523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:58.310338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:58.332113 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 984/1935] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:19:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:19:58.346166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:58.348136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:58.385995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998211fc-730e-4684-8d38-085fd52920fd admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:58.390967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:58.391629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.391889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.392813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] PATH_INFO: `/v3/projects/fcb757b16b4048c69cdbb2272dfed9c3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.394064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb757b16b4048c69cdbb2272dfed9c3)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:58.396145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:58.444799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2551de-e1e1-406e-9ada-70789efc6451 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:58.449718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:58.450426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.450779 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.451157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.464743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:58.474417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:58.486180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 985/1936] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:19:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:19:58.568388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb757b16b4048c69cdbb2272dfed9c3', 'request_id': 'req-2101cfe1-44cd-4543-a7ad-b3232a1b48af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:58.584157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb757b16b4048c69cdbb2272dfed9c3', 'request_id': 'req-2101cfe1-44cd-4543-a7ad-b3232a1b48af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:58.597671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb757b16b4048c69cdbb2272dfed9c3', 'request_id': 'req-2101cfe1-44cd-4543-a7ad-b3232a1b48af'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:58.678688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] The token cache is being invalidate because project fcb757b16b4048c69cdbb2272dfed9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:19:58.679177 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:58.680217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:58.682434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 952/1937] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:58 2022] DELETE /identity/v3/projects/fcb757b16b4048c69cdbb2272dfed9c3 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:58.686450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:58.719446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc5c48-3697-49eb-ab99-4bc233b17e37 service glance] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:19:58.846788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:19:58.849089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:19:58.849372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:19:58.849795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] PATH_INFO: `/v3/users/53921edda4a34513a676b205668d4f5e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:19:58.853205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53921edda4a34513a676b205668d4f5e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:19:58.856567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:19:59.009510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:19:59.087181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53921edda4a34513a676b205668d4f5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:59.095194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53921edda4a34513a676b205668d4f5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:59.095960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53921edda4a34513a676b205668d4f5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:59.163338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53921edda4a34513a676b205668d4f5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:59.168346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53921edda4a34513a676b205668d4f5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:19:59.221322 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 986/1938] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:19:58 2022] DELETE /identity/v3/users/53921edda4a34513a676b205668d4f5e => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:19:59.970352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:19:59.973006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:19:59.993833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101cfe1-44cd-4543-a7ad-b3232a1b48af admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:00.097660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:00.162603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:00.162603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:00.162603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:00.162603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] PATH_INFO: `/v3/projects/97d4eb46e41c46629c55105184549652` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:00.163113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d4eb46e41c46629c55105184549652)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:00.168238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:00.307383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d4eb46e41c46629c55105184549652', 'request_id': 'req-fd7056c4-3a51-4b09-af38-52ddba448054'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.324302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d4eb46e41c46629c55105184549652', 'request_id': 'req-fd7056c4-3a51-4b09-af38-52ddba448054'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.356775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d4eb46e41c46629c55105184549652', 'request_id': 'req-fd7056c4-3a51-4b09-af38-52ddba448054'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.466921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] The token cache is being invalidate because project 97d4eb46e41c46629c55105184549652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:20:00.467221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.468120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.470204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 953/1939] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:19:59 2022] DELETE /identity/v3/projects/97d4eb46e41c46629c55105184549652 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:00.473913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:00.476059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:00.495497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ff9b2-6835-4df6-963e-cc1c84b9c766 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:00.565728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:00.566687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:00.567242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:00.567820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] PATH_INFO: `/v3/users/8582c34067934b59b71cfd6befb7fee9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:00.569321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8582c34067934b59b71cfd6befb7fee9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:00.571414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:00.837814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8582c34067934b59b71cfd6befb7fee9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.848791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8582c34067934b59b71cfd6befb7fee9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.850175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8582c34067934b59b71cfd6befb7fee9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.913676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8582c34067934b59b71cfd6befb7fee9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.914843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7929815a-c464-4529-9bcf-ec67c31276a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8582c34067934b59b71cfd6befb7fee9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:00.965610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 987/1940] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:20:00 2022] DELETE /identity/v3/users/8582c34067934b59b71cfd6befb7fee9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:01.136721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:01.418762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:01.421103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:01.439045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7056c4-3a51-4b09-af38-52ddba448054 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:01.531243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:01.531898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:01.532181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:01.532504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] PATH_INFO: `/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:01.534120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1ce6b3c8df4f5999b8759f311abbfa)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:01.540528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:01.770800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1ce6b3c8df4f5999b8759f311abbfa', 'request_id': 'req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:01.783551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1ce6b3c8df4f5999b8759f311abbfa', 'request_id': 'req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:01.816316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1ce6b3c8df4f5999b8759f311abbfa', 'request_id': 'req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:01.924305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] The token cache is being invalidate because project fc1ce6b3c8df4f5999b8759f311abbfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:20:01.925133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:01.927079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3c6d2fb1-9765-4ca0-8e1a-ab5ba51e12e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:01.932863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 954/1941] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:20:01 2022] DELETE /identity/v3/projects/fc1ce6b3c8df4f5999b8759f311abbfa => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:01.938571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:01.939123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:01.939245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:01.969990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:01.971869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:02.075614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 988/1942] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:02.079439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.079439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.079682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.110894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:02.115764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:02.117310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:02.132277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.133268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 955/1943] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:02.145037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27847d-6a84-4bdd-ba8e-2f13de56f622 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.153007 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.153884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.154344 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.155030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.156634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:20:02.157166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.159012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.180384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.180932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 989/1944] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:20:02.207067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c67f-90cc-4e4e-ac2f-4489c5226450 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.229014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.229722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.230075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.230431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.231297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.233324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.332426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 956/1945] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:20:02 2022] POST /identity/v3/projects => generated 399 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:20:02.337198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.348438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92442f-13ba-4ac9-b7be-72483fd0b107 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.421855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.422568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.422942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.423596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.424163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.426190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.524916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 990/1946] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:20:02 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:20:02.529587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.540400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89369d4-d221-4a04-a1c6-5be7312874f9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.546007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.546600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.546957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.547232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.547869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.549749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.567315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 957/1947] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:20:02.568511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.582074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689499cb-bb1a-4470-9404-c2aea18b2d79 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.591635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.592418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.592763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.593144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.595443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.597435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c1c08-825c-49b5-bc76-c5a7f15130b5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.624427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 991/1948] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:20:02.628931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.640825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef17e22-92b1-4558-a6d1-21db279f9d58 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:02.644543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:02.672378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:02.674069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.675234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.675932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] PATH_INFO: `/v3/projects/087d975c398e4bf59293b8b106525489/users/390186056df64d1c822ff5534565fe9d/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.702309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087d975c398e4bf59293b8b106525489, user_id=390186056df64d1c822ff5534565fe9d, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:02.707539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:02.791228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 958/1949] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:20:02 2022] PUT /identity/v3/projects/087d975c398e4bf59293b8b106525489/users/390186056df64d1c822ff5534565fe9d/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:02.802135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f8c5-f5ae-474a-ad58-1c323c65f967 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:02.802630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f8c5-f5ae-474a-ad58-1c323c65f967 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:02.802775 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f8c5-f5ae-474a-ad58-1c323c65f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:02.855008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-95e5f8c5-f5ae-474a-ad58-1c323c65f967 None None] MFA Rules not processed for user `390186056df64d1c822ff5534565fe9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:02.857105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-95e5f8c5-f5ae-474a-ad58-1c323c65f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:02.969835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 992/1950] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:02.986614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:02.988447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:03.100668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d5588-6e2f-4afb-aef3-1fa443ca7ff7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:03.107106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:03.108108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:03.111945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:03.111945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:03.127290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:03.129043 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:03.143427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 959/1951] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:20:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:20:03.822470 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:03.822958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:03.823279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:03.856269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] MFA Rules not processed for user `28e0de7076904034a98885c8ebeddce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:03.857607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:03.941511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 993/1952] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:03.957999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:03.960653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:04.041604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283ca82-d140-49d0-a603-e4691e0a9d70 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:04.047616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:04.048632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:04.048770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:04.049199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:04.050991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:04.090251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:04.125542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:04.127187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941813-11cf-4e23-a8ae-2e8dc470d82b service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:04.144798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 960/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:20:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:20:04.334576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:15.420115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=13 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:15.616823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=25 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:26.029651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:26.125791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8581e-3732-416b-afbb-e46c7732291a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:26.130419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:26.131107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:26.131387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:26.131692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] PATH_INFO: `/v3/users/390186056df64d1c822ff5534565fe9d` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:26.132858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=390186056df64d1c822ff5534565fe9d)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:26.134841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:26.319365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390186056df64d1c822ff5534565fe9d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:26.326290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390186056df64d1c822ff5534565fe9d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:26.327091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390186056df64d1c822ff5534565fe9d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:26.413111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390186056df64d1c822ff5534565fe9d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:26.413806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390186056df64d1c822ff5534565fe9d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:26.434511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 994/1954] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:20:26 2022] DELETE /identity/v3/users/390186056df64d1c822ff5534565fe9d => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:26.439546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:26.440044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:26.440157 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:26.461201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:26.462406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:26.570871 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 961/1955] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:26.584367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:26.585783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:26.646132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1450258-aec1-484d-ba83-ed923571d647 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:26.649907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:26.650527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:26.650970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:26.651270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:26.665154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:26.666670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f6cc1-7b02-49fe-aa6e-9795bdbd2c38 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:26.679201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 995/1956] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:20:27.276865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:20:27.276865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:27.303198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ababe-831b-4884-9528-19d9c4239728 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:20:27.309013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:20:27.309750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:27.310483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:27.311016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] PATH_INFO: `/v3/projects/087d975c398e4bf59293b8b106525489` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:27.312337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087d975c398e4bf59293b8b106525489)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:20:27.316275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:20:27.432576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087d975c398e4bf59293b8b106525489', 'request_id': 'req-ef54408b-2684-47ee-b650-92d3c93a3753'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:27.440171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087d975c398e4bf59293b8b106525489', 'request_id': 'req-ef54408b-2684-47ee-b650-92d3c93a3753'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:27.446397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087d975c398e4bf59293b8b106525489', 'request_id': 'req-ef54408b-2684-47ee-b650-92d3c93a3753'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:27.517202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] The token cache is being invalidate because project 087d975c398e4bf59293b8b106525489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:20:27.517499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:27.518138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ef54408b-2684-47ee-b650-92d3c93a3753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:20:27.519884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 962/1957] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:20:27 2022] DELETE /identity/v3/projects/087d975c398e4bf59293b8b106525489 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:20:30.285502 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aaeb1-c4d7-4319-b9db-e5fd29010590 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:30.285997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aaeb1-c4d7-4319-b9db-e5fd29010590 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:30.286361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920aaeb1-c4d7-4319-b9db-e5fd29010590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:30.305927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-920aaeb1-c4d7-4319-b9db-e5fd29010590 None None] MFA Rules not processed for user `a79972df53cf4245be9c45a9b579cff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:30.307285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-920aaeb1-c4d7-4319-b9db-e5fd29010590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:30.324609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:30.363539 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 996/1958] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:32.854926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:32.855380 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:32.855478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:32.878358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] MFA Rules not processed for user `ff0c5f8521534fc1a0a9c5ea28fbaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:32.879677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:32.946415 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 963/1959] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:41.013786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=15 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:42.939062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:20:48.835627 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:20:48.836177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:20:48.836177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:20:48.856260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] MFA Rules not processed for user `1f737b4d0531425cbc803fcd15a3ea1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:20:48.857469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:20:48.921403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 997/1960] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:20:58.916478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=25 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:03.008919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:03.010936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:03.071293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fcfe8-403c-4852-b1ca-4454326c6ecc None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:03.075798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:03.076418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:03.076723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:03.076994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:03.078810 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:03.146321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:03.147790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197fd2b-26f5-45b6-a5fe-805b57a2c702 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:03.157958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 964/1961] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:06.571196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:06.573631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:06.683885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36b110-3fa4-4b47-88da-f1a72ebb8e97 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:06.689335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:06.690173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:06.690447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:06.690902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] PATH_INFO: `/v3/users/f6f611c8544040439f53e1ad2e98ebf7` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:06.692564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f611c8544040439f53e1ad2e98ebf7)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:06.694639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:06.949229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f611c8544040439f53e1ad2e98ebf7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:06.957457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f611c8544040439f53e1ad2e98ebf7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:06.959810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f611c8544040439f53e1ad2e98ebf7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:07.036459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f611c8544040439f53e1ad2e98ebf7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:07.037289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f611c8544040439f53e1ad2e98ebf7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:07.062252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 998/1962] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:21:06 2022] DELETE /identity/v3/users/f6f611c8544040439f53e1ad2e98ebf7 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:07.066970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:07.067478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:07.068402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:07.090142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:07.091927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:07.160210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 965/1963] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:07.201774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:07.203802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:07.295816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d01b0-db5f-4d4c-b83d-04b797134e07 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:07.305552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:07.306317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:07.310947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:07.310947 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:07.319431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:07.321019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2cbdda-4b71-40d1-b601-b5cb2c52dfc5 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:07.333472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 999/1964] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:08.812520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:08.813722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:08.827436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf23e2-2062-4e54-af67-a10c9be60ffa None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:08.832626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:08.833268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:08.833564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:08.833839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] PATH_INFO: `/v3/projects/d34ea9c8581c4a53a949b046a7776789` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:08.835055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34ea9c8581c4a53a949b046a7776789)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:08.836955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:08.950053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:08.952558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34ea9c8581c4a53a949b046a7776789', 'request_id': 'req-253a352d-71a5-4ac3-b073-16ba6ccbf013'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:08.960444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34ea9c8581c4a53a949b046a7776789', 'request_id': 'req-253a352d-71a5-4ac3-b073-16ba6ccbf013'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:08.966109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34ea9c8581c4a53a949b046a7776789', 'request_id': 'req-253a352d-71a5-4ac3-b073-16ba6ccbf013'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:09.043154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] The token cache is being invalidate because project d34ea9c8581c4a53a949b046a7776789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:21:09.043460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:09.044115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-253a352d-71a5-4ac3-b073-16ba6ccbf013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:09.049072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 966/1965] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:21:08 2022] DELETE /identity/v3/projects/d34ea9c8581c4a53a949b046a7776789 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:09.050655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.051147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.051231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.073242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:09.074644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:09.093254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:09.141461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1000/1966] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:09.150425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.150709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.151550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.174603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:09.175850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:09.183959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 967/1967] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:09.188054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.199073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eeedf-2b89-4078-9dff-786aae206cd6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.204593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.205043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.209459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.209459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.209459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:21:09.209459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.209459 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.222842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1001/1968] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:21:09.226768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.236726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5077-7157-4d09-ab10-99673ca54f9d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.241946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.242851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.243154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.243987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.244105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.246367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.356954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 968/1969] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:21:09 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:21:09.358441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.377761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b8d3f-581a-4d17-a043-dbe8eacc2104 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.461610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.462318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.462575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.462872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.464066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.466045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.800060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1002/1970] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:21:09 2022] POST /identity/v3/users => generated 286 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:21:09.801846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.813464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fd3e2-4222-4478-9471-f4e4ff761090 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.823421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.823421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.823421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.823421 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.824023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.824023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.838013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 969/1971] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:21:09.842700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.853043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b0edb-1153-4c66-a3de-92fda23808d5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.859704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.860610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.861183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.861183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.861657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.863289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9a6ff-d2b7-4042-b4f6-8260ff51884c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.876334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1003/1972] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:21:09.880648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:09.891742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5099bf6c-b351-43eb-8599-81710cd00558 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:09.896539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:09.901671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.901671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.901671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] PATH_INFO: `/v3/projects/3c55d5d8b72b4cf3b348995f49e96c5f/users/3ddef6f6009e4c638b3c0f0cc2d4ccd9/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:09.914447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c55d5d8b72b4cf3b348995f49e96c5f, user_id=3ddef6f6009e4c638b3c0f0cc2d4ccd9, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:09.917718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:09.980730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 970/1973] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:21:09 2022] PUT /identity/v3/projects/3c55d5d8b72b4cf3b348995f49e96c5f/users/3ddef6f6009e4c638b3c0f0cc2d4ccd9/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:09.991530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501a9e-5d9f-4f0e-a903-71233a04e29c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:09.991828 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501a9e-5d9f-4f0e-a903-71233a04e29c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:09.992063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81501a9e-5d9f-4f0e-a903-71233a04e29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:10.022810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-81501a9e-5d9f-4f0e-a903-71233a04e29c None None] MFA Rules not processed for user `3ddef6f6009e4c638b3c0f0cc2d4ccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:10.024039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-81501a9e-5d9f-4f0e-a903-71233a04e29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:10.103732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1004/1974] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:10.114952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:10.116712 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:10.184258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2df702-7537-4b9e-ae93-9040da24e29f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:10.188306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:10.188959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:10.189244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:10.189571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:10.200660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:10.202142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef84a5-8a2c-4393-ac3d-3d26a236157e service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:10.212445 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 971/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:12.949705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60b39b-fb76-449c-b964-d5cb7bdaae71 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:12.950190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60b39b-fb76-449c-b964-d5cb7bdaae71 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:12.950314 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60b39b-fb76-449c-b964-d5cb7bdaae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:12.968759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-fd60b39b-fb76-449c-b964-d5cb7bdaae71 None None] MFA Rules not processed for user `3ddef6f6009e4c638b3c0f0cc2d4ccd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:12.969944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fd60b39b-fb76-449c-b964-d5cb7bdaae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:12.976630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1005/1976] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:16.539126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612688-68ef-4949-baa6-aa1f34df6d16 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:16.539575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612688-68ef-4949-baa6-aa1f34df6d16 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:16.539708 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612688-68ef-4949-baa6-aa1f34df6d16 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:16.541102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 972/1977] 10.222.0.42 () {54 vars in 961 bytes} [Wed Oct 26 11:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:21:16.546620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cdbf-9b77-4c2e-a522-1fb87fa8a0b0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:16.547073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cdbf-9b77-4c2e-a522-1fb87fa8a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:16.547186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cdbf-9b77-4c2e-a522-1fb87fa8a0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:16.566478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8454cdbf-9b77-4c2e-a522-1fb87fa8a0b0 None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:16.567952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8454cdbf-9b77-4c2e-a522-1fb87fa8a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:16.573167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:16.631595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1006/1978] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Oct 26 11:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:16.639542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee612688-68ef-4949-baa6-aa1f34df6d16 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:16.652729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee612688-68ef-4949-baa6-aa1f34df6d16 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:16.657588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:16.658184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:16.658461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:16.658730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:16.660469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:16.672548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:16.674092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:16.684120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 973/1979] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:17.764648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:17.774847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:17.774847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:17.784580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:17.785750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:17.843111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1007/1980] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:21:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:17.886755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:17.888375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:17.956159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:17.966181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee741d-dc15-41d3-8304-feac2041740e service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:17.978961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:17.978961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:17.978961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:17.978961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:17.982031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:17.983649 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50fe177-d5f2-4c8d-be04-dd6979352eb6 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:17.992246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 974/1981] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:21:27.836679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=49 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:28.044845 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:30.524842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:30.526296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:30.588712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa390f-fff5-46b6-b999-3d8b83470f58 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:30.593521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:30.594138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:30.594420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:30.594696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:30.596542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:30.677060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:30.679058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:30.690120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1008/1982] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:36.335831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04688-a9c3-4e14-ba50-8cfe2de06c19 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:36.336277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04688-a9c3-4e14-ba50-8cfe2de06c19 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:36.336369 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b04688-a9c3-4e14-ba50-8cfe2de06c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:36.356273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-76b04688-a9c3-4e14-ba50-8cfe2de06c19 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:36.357507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-76b04688-a9c3-4e14-ba50-8cfe2de06c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:36.365938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 975/1983] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:21:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:36.378966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:36.389897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbe6f-d01f-4d5c-8fc3-fd73f66eac5e service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:36.394866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:36.395469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:36.395759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:36.396020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:36.404212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:36.406283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854b8f-f2e7-4fc6-86d3-d49dcbad0da4 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:36.414500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1009/1984] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:21:39.429717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:39.430216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:39.430333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:39.431766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 976/1985] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:21:39.442452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:39.442910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:39.443032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:39.462997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] MFA Rules not processed for user `e4a26f6a0712462996a17c7fe00a68ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:39.464214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:39.473857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1010/1986] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:21:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:39.485102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:39.486958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:39.502961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c5684-deac-43b7-94d8-367c2b4bfae6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:39.507868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:39.508489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:39.508765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:39.509859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:39.517675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:39.519295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:39.528746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 977/1987] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:39.558486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:39.559942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:39.628588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0dcec-4a04-4535-aecf-5bfd4cd45b9b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:39.633212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:39.633881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:39.634159 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:39.634435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] PATH_INFO: `/v3/users/a79972df53cf4245be9c45a9b579cff9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:39.635593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79972df53cf4245be9c45a9b579cff9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:39.637518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:39.797384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:39.809375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79972df53cf4245be9c45a9b579cff9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:39.816880 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79972df53cf4245be9c45a9b579cff9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:39.817555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79972df53cf4245be9c45a9b579cff9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:39.887123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79972df53cf4245be9c45a9b579cff9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:39.887762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5bd7d1b7-bf37-4e1a-a5af-9f9f9901f62f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79972df53cf4245be9c45a9b579cff9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:39.908318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1011/1988] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:21:39 2022] DELETE /identity/v3/users/a79972df53cf4245be9c45a9b579cff9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:40.180282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:40.191744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fbc00-cc9e-4d77-ae50-4b43cb9b5c81 service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:40.249264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:40.249890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.250170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.251112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] PATH_INFO: `/v3/projects/8576eb9289f147f2ab9ba6f1c00b3ffd` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.252208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8576eb9289f147f2ab9ba6f1c00b3ffd)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:40.254082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:40.361155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8576eb9289f147f2ab9ba6f1c00b3ffd', 'request_id': 'req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:40.367618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8576eb9289f147f2ab9ba6f1c00b3ffd', 'request_id': 'req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:40.373163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8576eb9289f147f2ab9ba6f1c00b3ffd', 'request_id': 'req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:40.476294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] The token cache is being invalidate because project 8576eb9289f147f2ab9ba6f1c00b3ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:21:40.476930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:40.477350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a654fc00-a1d5-49e3-9de2-ed1f294e55e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:40.478934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 978/1989] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:21:40 2022] DELETE /identity/v3/projects/8576eb9289f147f2ab9ba6f1c00b3ffd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:40.484630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.485583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.485919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.507751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:40.508989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:40.569693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1012/1990] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:21:40.577601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.577887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.578159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.598584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:40.600101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:40.609119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 979/1991] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:40.614736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:40.624453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe7388-ffa9-46e1-a2fe-1f0d3a7505fb None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:40.629772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:40.630384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.630665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.630979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.631626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:21:40.631931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:40.633649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:40.645926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1013/1992] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:21:40.650194 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:40.661079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf8e1c-b24f-41a1-8f17-cd56feece1aa None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:40.668707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:40.668707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.668707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.668707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.669115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:40.670132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:40.815053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 980/1993] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:21:40 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:21:40.819290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:40.829574 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c8d5-9401-44d5-afc2-2e1c690b44c3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:40.884927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:40.885529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:40.885808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:40.886080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:40.886705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:40.888647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.003144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1014/1994] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:21:40 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:21:41.011390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:41.022625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17602e3-8a21-4de2-b7c8-d838e32d052d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:41.027660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:41.028284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.028563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.028838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.029494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:41.031170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.041922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 981/1995] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:21:41.045876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:41.055901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617b5f2-6189-43bb-8e6a-c8f4efe8361b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:41.060515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:41.061107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.061390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.061665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.062300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:41.063955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f991-7e38-42f2-93f0-a28eb4c22698 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.074937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1015/1996] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:21:41.079024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:41.089523 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff5c7e-abf6-46bf-aeb7-abac609cb613 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:41.094173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:41.094839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.095752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.096034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] PATH_INFO: `/v3/projects/f53873c9b2fc493496a8665e70546b1d/users/b1d5b3c8ce5d46698f0f02704921d906/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.110803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53873c9b2fc493496a8665e70546b1d, user_id=b1d5b3c8ce5d46698f0f02704921d906, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:41.113525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f7d80-ef40-45ae-85ab-23e866206b37 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.189012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 982/1997] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:21:41 2022] PUT /identity/v3/projects/f53873c9b2fc493496a8665e70546b1d/users/b1d5b3c8ce5d46698f0f02704921d906/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:41.199450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.199734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.200008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.230092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] MFA Rules not processed for user `b1d5b3c8ce5d46698f0f02704921d906`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:41.231266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:41.235164 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:41.263016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.263301 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.263577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.278092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:41.284687 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:41.285914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:41.299841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1016/1998] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:41.311315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:41.314444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:41.351084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 983/1999] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:21:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:41.364604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:41.366091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:41.375229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e51d1e-0fd5-4420-a9ab-abdff90a8f43 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:41.378951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:41.379558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.379835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.380103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.388387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:41.389842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1418-4a52-46f3-952c-c4173bd8e645 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.398547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1017/2000] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:21:41.436258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898570a-6c3a-4b79-bb71-7bb5753b3744 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:41.442413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:41.443283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:41.443562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:41.443836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:41.452241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:41.453847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:41.461931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 984/2001] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:21:42.402926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1e6e-1c0b-4e7d-a864-78b796afb2ad None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:42.403440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1e6e-1c0b-4e7d-a864-78b796afb2ad None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:42.403541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1e6e-1c0b-4e7d-a864-78b796afb2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:42.422854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-29ad1e6e-1c0b-4e7d-a864-78b796afb2ad None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:42.424523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-29ad1e6e-1c0b-4e7d-a864-78b796afb2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:42.431599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1018/2002] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:21:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:42.446937 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:42.456681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1641f6b-84d1-4e93-89e6-a24c6e09cc1f service swift] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:42.461600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:42.462208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:42.462475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:42.462928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:42.471420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:42.473212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:42.481260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 985/2003] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:21:43.118083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e300-7b1e-4f43-a0e7-0233e0de53a5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:43.118370 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e300-7b1e-4f43-a0e7-0233e0de53a5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:43.118644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2e300-7b1e-4f43-a0e7-0233e0de53a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:43.137914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-91a2e300-7b1e-4f43-a0e7-0233e0de53a5 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:43.139135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-91a2e300-7b1e-4f43-a0e7-0233e0de53a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:43.146659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1019/2004] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:21:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:43.159449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:43.169667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cdcfd-5609-427e-a1f1-1acf03b5a669 service swift] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:43.177659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:43.178280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:43.178559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:43.178855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:43.187240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:43.188762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:43.197125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 986/2005] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:21:43.873748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3490-f1c1-4d07-8e81-78567b506e63 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:43.874207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3490-f1c1-4d07-8e81-78567b506e63 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:43.874348 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3490-f1c1-4d07-8e81-78567b506e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:43.900944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-055a3490-f1c1-4d07-8e81-78567b506e63 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:43.902163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-055a3490-f1c1-4d07-8e81-78567b506e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:43.910054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1020/2006] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:21:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:43.927726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:43.942300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe755fe3-87ef-4772-93b2-b126c2dfe44b service swift] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:43.948045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:43.948598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:43.948879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:43.949280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:43.958377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:43.960205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:43.968792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 987/2007] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:21:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:21:44.468350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c15d2b-9bd4-495d-b762-65cbed5d4081 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:44.468350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c15d2b-9bd4-495d-b762-65cbed5d4081 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:44.468350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c15d2b-9bd4-495d-b762-65cbed5d4081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:44.497270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d0c15d2b-9bd4-495d-b762-65cbed5d4081 None None] MFA Rules not processed for user `e031dd9da25146eebb8813dc8bad8b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:44.498602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d0c15d2b-9bd4-495d-b762-65cbed5d4081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:44.573632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1021/2008] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:45.546456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:45.614173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42b08b-6d42-492d-9723-34a3cd4e8ed3 service swift] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:45.619496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:45.620141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:45.620461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:45.620705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] PATH_INFO: `/v3/users/b1d5b3c8ce5d46698f0f02704921d906` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:45.621832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d5b3c8ce5d46698f0f02704921d906)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:45.623824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:45.626469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:45.913910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d5b3c8ce5d46698f0f02704921d906'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:45.921533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d5b3c8ce5d46698f0f02704921d906'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:45.922221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d5b3c8ce5d46698f0f02704921d906'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.009053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d5b3c8ce5d46698f0f02704921d906'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.009696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d5b3c8ce5d46698f0f02704921d906'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.029562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 988/2009] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:21:45 2022] DELETE /identity/v3/users/b1d5b3c8ce5d46698f0f02704921d906 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:46.034840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:46.035284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:46.035533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:46.055028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:46.056220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:46.084701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:46.114176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1022/2010] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:46.124084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:46.125590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:46.182614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a1bc9-94d3-40ce-b72e-dfbc0ddfdf41 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:46.186054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:46.186652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:46.187024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:46.187297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:46.196185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:46.197847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:46.207684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 989/2011] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:46.667254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:46.675838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:46.675982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:46.675982 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:46.685871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64dc68-5dd7-4f83-b140-8782831e10e2 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:46.696707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e514723-b775-4547-8a52-9240ba4d6885 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:46.702643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:46.703498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:46.703921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:46.709447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] PATH_INFO: `/v3/projects/f53873c9b2fc493496a8665e70546b1d` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:46.709447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53873c9b2fc493496a8665e70546b1d)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:46.709615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:46.709615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:46.709615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:46.709615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] PATH_INFO: `/v3/users/3ddef6f6009e4c638b3c0f0cc2d4ccd9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:46.709949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:46.710693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ddef6f6009e4c638b3c0f0cc2d4ccd9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:46.712961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:46.891280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ddef6f6009e4c638b3c0f0cc2d4ccd9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.892470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53873c9b2fc493496a8665e70546b1d', 'request_id': 'req-1586b4a7-3e80-4233-b269-23a916e34b40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.899187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ddef6f6009e4c638b3c0f0cc2d4ccd9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.899342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53873c9b2fc493496a8665e70546b1d', 'request_id': 'req-1586b4a7-3e80-4233-b269-23a916e34b40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.899862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ddef6f6009e4c638b3c0f0cc2d4ccd9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.905036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53873c9b2fc493496a8665e70546b1d', 'request_id': 'req-1586b4a7-3e80-4233-b269-23a916e34b40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.973382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ddef6f6009e4c638b3c0f0cc2d4ccd9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.974028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb9b1d8a-deb2-4661-bfd8-02c1402ac61a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ddef6f6009e4c638b3c0f0cc2d4ccd9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:46.994447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1023/2012] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:21:46 2022] DELETE /identity/v3/users/3ddef6f6009e4c638b3c0f0cc2d4ccd9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:46.998590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:46.998901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:46.999181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.019523 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:47.020716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.034034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] The token cache is being invalidate because project f53873c9b2fc493496a8665e70546b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:21:47.034220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.035297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1586b4a7-3e80-4233-b269-23a916e34b40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.037516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 990/2013] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:21:46 2022] DELETE /identity/v3/projects/f53873c9b2fc493496a8665e70546b1d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:21:47.042224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402813-fc04-43c0-822f-78f9682372c2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.042583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402813-fc04-43c0-822f-78f9682372c2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.042994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e402813-fc04-43c0-822f-78f9682372c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.061601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:47.067974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1e402813-fc04-43c0-822f-78f9682372c2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:47.069250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1e402813-fc04-43c0-822f-78f9682372c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.093790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1024/2014] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:47.103942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.105389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.140948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 991/2015] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:21:47.150746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.150746 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.151041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.165511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:47.172008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:47.173503 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.173769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6feb47-9f6b-45e9-8181-b3ab7fdeffa1 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.179188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.180034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.180495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.181109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.187139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 992/2016] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:47.188055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.195565 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.197324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.203820 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc66cfb-6656-45ba-a103-96b1dc317fa0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.208306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1025/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:47.212011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.212638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.213120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.213748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.214459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:21:47.214767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.216412 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.243840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 993/2018] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:21:47.245009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.255907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace8b0a-8a0c-4aed-a23f-79934898fd5e service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.261570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.264633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.264633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.264633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.264633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.266479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.360545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1026/2019] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:21:47 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:21:47.361592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.372693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd002c-4357-4bde-b9be-ea00beb5b83b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.432898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.433511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.433801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.434154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.434779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.436829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.528433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 994/2020] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:21:47 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:21:47.532579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.543171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e8240-1329-438d-9b9f-f4b483852df1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.548789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.549391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.549713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.549924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.551176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.552752 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.561524 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.563602 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.564679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1027/2021] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:21:47.568866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.580406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9946e-d35f-4d8f-aff1-385868dbef1f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.587751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.588300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.588670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.589018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] PATH_INFO: `/v3/projects/3c55d5d8b72b4cf3b348995f49e96c5f` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.589980 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957bddc-91db-42c9-ab2d-f5799341168a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.591159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c55d5d8b72b4cf3b348995f49e96c5f)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.593730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.595568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.596233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.598151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.598529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.599720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.601398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.614271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1028/2022] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:21:47.618964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.629721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcb495-2366-4e47-a3cc-6a8587cfc75b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.634473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.635170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.635450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.635721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] PATH_INFO: `/v3/projects/6490ce5a37be456db0da974d3e21ffcc/users/73134ccef60442128680e577a7ba5235/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.651568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6490ce5a37be456db0da974d3e21ffcc, user_id=73134ccef60442128680e577a7ba5235, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.654313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec07bc7-5a6f-4c56-b07d-6100c629ca7c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.743664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1029/2023] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:21:47 2022] PUT /identity/v3/projects/6490ce5a37be456db0da974d3e21ffcc/users/73134ccef60442128680e577a7ba5235/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:21:47.749878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.753423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.753423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.760315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c55d5d8b72b4cf3b348995f49e96c5f', 'request_id': 'req-64c1c191-6eb7-41c4-a355-a94d0a75155d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.767478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c55d5d8b72b4cf3b348995f49e96c5f', 'request_id': 'req-64c1c191-6eb7-41c4-a355-a94d0a75155d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.774012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c55d5d8b72b4cf3b348995f49e96c5f', 'request_id': 'req-64c1c191-6eb7-41c4-a355-a94d0a75155d'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.780587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] MFA Rules not processed for user `73134ccef60442128680e577a7ba5235`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:21:47.781851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.843957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:21:47.848934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1030/2024] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:21:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:21:47.849088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] The token cache is being invalidate because project 3c55d5d8b72b4cf3b348995f49e96c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:21:47.849211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.849932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-64c1c191-6eb7-41c4-a355-a94d0a75155d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:21:47.851723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 995/2025] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:21:47 2022] DELETE /identity/v3/projects/3c55d5d8b72b4cf3b348995f49e96c5f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:21:47.860352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:21:47.863219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.935151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90adeb5-c4c5-4260-a5b8-002a843ca8bd None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:21:47.939270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:21:47.940282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:21:47.940756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:21:47.941209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:21:47.943612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:21:47.957136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:21:47.958590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:21:47.969791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1031/2026] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:21:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:21:57.964187 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:14.049156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12eb5-7113-468b-9e6f-30f007b3fefa None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:14.049689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12eb5-7113-468b-9e6f-30f007b3fefa None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:14.049787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf12eb5-7113-468b-9e6f-30f007b3fefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:14.070540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-dcf12eb5-7113-468b-9e6f-30f007b3fefa None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:14.074504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dcf12eb5-7113-468b-9e6f-30f007b3fefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:14.126507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 996/2027] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:22:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:14.140759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:14.142166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:14.211956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0971cc-bb30-4a6d-acc7-948232352031 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:14.217761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:14.218489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:14.218768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:14.219076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:14.227608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:14.229460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d970c19-4e26-4660-9e42-e336fd5ba104 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:14.239338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1032/2028] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:22:24.119716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=25 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:24.235298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=32 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:34.150414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.150909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.151099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.152449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 997/2029] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 26 11:22:34.158540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf4c2-2057-49fd-b99e-4266729bddf6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.159080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf4c2-2057-49fd-b99e-4266729bddf6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.159171 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00abf4c2-2057-49fd-b99e-4266729bddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.179695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-00abf4c2-2057-49fd-b99e-4266729bddf6 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:34.180972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-00abf4c2-2057-49fd-b99e-4266729bddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:34.252509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1033/2030] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:34.263855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:34.265582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:34.323091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206bca3-7e2c-4763-9ef2-8ed5b0a2320e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:34.326520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:34.327145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.327427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.327701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.337736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:34.339189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc3dcb-b225-428b-83c3-8e3d5896a04b service glance] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:34.350377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 998/2031] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:22:34.503912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.504194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.504461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.505808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1034/2032] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:22:34.510609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e26b-6565-4be3-bc54-b5f1b8d9f498 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.511018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e26b-6565-4be3-bc54-b5f1b8d9f498 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.511296 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e26b-6565-4be3-bc54-b5f1b8d9f498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.530723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c025e26b-6565-4be3-bc54-b5f1b8d9f498 None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:34.531957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c025e26b-6565-4be3-bc54-b5f1b8d9f498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:34.582300 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 999/2033] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:34.593759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:34.595936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:34.608583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2366f-fc56-44b9-acf1-d08cfbc41e1b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:34.612655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:34.613232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.613516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.613901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.620694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:34.622088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd63c66-79fb-4695-b869-8a38b007a73d service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:34.629629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1035/2034] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:22:34.711450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a447a-ab16-49a3-90c8-0994013ccbff None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.711737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a447a-ab16-49a3-90c8-0994013ccbff None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.712011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a447a-ab16-49a3-90c8-0994013ccbff None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.713376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1000/2035] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:22:34.718144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1facf1-97c7-46c9-b47b-fd3ab6193fd3 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.718440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1facf1-97c7-46c9-b47b-fd3ab6193fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.718712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1facf1-97c7-46c9-b47b-fd3ab6193fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.740117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1f1facf1-97c7-46c9-b47b-fd3ab6193fd3 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:34.741307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1f1facf1-97c7-46c9-b47b-fd3ab6193fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:34.752691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1036/2036] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:34.763061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a447a-ab16-49a3-90c8-0994013ccbff None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:34.774413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a447a-ab16-49a3-90c8-0994013ccbff None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:34.777965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:34.778552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:34.778855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:34.779137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:34.789522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:34.790967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:34.802167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1001/2037] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:22:38.842107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:38.842578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:38.842671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:38.863060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:38.864293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:38.871330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1037/2038] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:38.883849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:38.895210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396089d4-cfda-402a-aac9-5e82320ffb44 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:38.900314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:38.901030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:38.901323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:38.901597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:38.910288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:38.916520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78e338-9b8d-41ef-97de-b82e11f00696 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:38.920058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1002/2039] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:22:45.828718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:45.831631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:45.900594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4be8d-431d-4e79-a575-29c1461e69f6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:45.905591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:45.906305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:45.906527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:45.906798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] PATH_INFO: `/v3/users/73134ccef60442128680e577a7ba5235` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:45.907961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=73134ccef60442128680e577a7ba5235)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:45.910558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:46.092802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73134ccef60442128680e577a7ba5235'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:46.100230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73134ccef60442128680e577a7ba5235'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:46.100945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73134ccef60442128680e577a7ba5235'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:46.174318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73134ccef60442128680e577a7ba5235'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:46.175103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73134ccef60442128680e577a7ba5235'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:46.180939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:46.195636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1038/2040] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:22:45 2022] DELETE /identity/v3/users/73134ccef60442128680e577a7ba5235 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:22:46.200172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:46.200700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:46.200700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:46.220433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:46.221672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:46.300683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1003/2041] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:46.310412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:46.312397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:46.404972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb79ce-9fa0-44d3-a4f9-2a07378699ec admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:46.408689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:46.409384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:46.409728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:46.410069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:46.423696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:46.423696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68398e3-d9fe-4202-8bc7-473f68e0f899 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:46.433744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1039/2042] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:22:46.868148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:46.869734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:46.883622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e700b5b-5187-4045-a14b-2d13c537d77a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:46.890050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:46.890917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:46.891321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:46.891660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] PATH_INFO: `/v3/projects/6490ce5a37be456db0da974d3e21ffcc` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:46.892942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6490ce5a37be456db0da974d3e21ffcc)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:46.894991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:46.925479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:47.009320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6490ce5a37be456db0da974d3e21ffcc', 'request_id': 'req-7945973b-dd57-4e04-8e09-a825fd8c47d2'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:47.016894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6490ce5a37be456db0da974d3e21ffcc', 'request_id': 'req-7945973b-dd57-4e04-8e09-a825fd8c47d2'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:47.022952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6490ce5a37be456db0da974d3e21ffcc', 'request_id': 'req-7945973b-dd57-4e04-8e09-a825fd8c47d2'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:47.111002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] The token cache is being invalidate because project 6490ce5a37be456db0da974d3e21ffcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:22:47.111395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:47.111980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7945973b-dd57-4e04-8e09-a825fd8c47d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:47.113925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1004/2043] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:22:46 2022] DELETE /identity/v3/projects/6490ce5a37be456db0da974d3e21ffcc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:22:47.118884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.119978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.120079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.140494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:47.141739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:47.202564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1040/2044] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:47.210058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.210761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.210761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.233617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:47.234897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:47.242789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1005/2045] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:47.257382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.272113 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21065d-b7d5-4602-ae10-64fa3d510f0a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.279064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.279064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.279064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.279064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.279461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:22:47.279461 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.281009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.293058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1041/2046] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:22:47.297387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.307662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917ec9-0c70-427e-b497-5668cbab59a7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.314835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.314835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.314835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.314835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.315219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.316142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.399992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1006/2047] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:22:47 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:22:47.404067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.414492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75827-facd-4003-a610-f3cdf2dd854d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.480763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.481315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.481595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.481871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.482516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.484480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.596041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1042/2048] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:22:47 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:22:47.600668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.609175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33520b0f-26b9-4672-92c5-0609d6c85930 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.613846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.614628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.614941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.615221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.615861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.617458 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.633476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.641153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:47.641361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1007/2049] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:22:47.645405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfc481-e19a-4982-a9ac-b6044c57a157 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.650686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.651340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.659339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.659339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.659339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.659339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03464a-7921-4124-84d7-5d6018497876 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.666495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1043/2050] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:22:47.670721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.681400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a2d899-2a55-41e1-847a-7834f8c40816 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.686678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.687648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.687761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.688070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] PATH_INFO: `/v3/projects/a8d0005c494b4c6a99f2bc7fee9bd532/users/c8527ec1b7fb411a9824d7011b80af43/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.704411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d0005c494b4c6a99f2bc7fee9bd532, user_id=c8527ec1b7fb411a9824d7011b80af43, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.707224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.771109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1008/2051] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:22:47 2022] PUT /identity/v3/projects/a8d0005c494b4c6a99f2bc7fee9bd532/users/c8527ec1b7fb411a9824d7011b80af43/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:22:47.782468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.782755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.783077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.822167 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] MFA Rules not processed for user `c8527ec1b7fb411a9824d7011b80af43`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:47.823427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:47.899686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1044/2052] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:47.907464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:47.909272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:47.968685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5da074-2ee9-4ded-9436-ddcac3ba1158 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:47.972817 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:47.973462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:47.973742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:47.974019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:47.983386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:47.984836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833a212-c4dc-4083-bfac-c57db05315a5 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:47.995147 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1009/2053] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:22:48.022060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:48.088244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b00ebf-fed1-46cf-82c5-452d22cbd607 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:48.092824 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:48.093465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:48.093748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:48.094023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] PATH_INFO: `/v3/users/c8527ec1b7fb411a9824d7011b80af43` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:48.095232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8527ec1b7fb411a9824d7011b80af43)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:48.097183 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:48.247081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8527ec1b7fb411a9824d7011b80af43'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:48.254137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8527ec1b7fb411a9824d7011b80af43'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:48.254755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8527ec1b7fb411a9824d7011b80af43'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:48.304816 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8527ec1b7fb411a9824d7011b80af43'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:48.305449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8527ec1b7fb411a9824d7011b80af43'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:48.327615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1045/2054] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:22:48 2022] DELETE /identity/v3/users/c8527ec1b7fb411a9824d7011b80af43 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:22:48.329284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:48.329838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:48.329838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:48.350769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:22:48.352044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:48.395898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:48.411666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1010/2055] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:22:48.421423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:48.423112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:48.428764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:48.483804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac735-f546-42ac-80ed-6ff0d7637175 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:48.487427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:48.488038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:48.488317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:48.488585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:48.498298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:48.500080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:48.511650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1046/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:22:48.963765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:48.965335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:48.979392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530cefb-6bec-4667-b5e1-759a01c2fdae None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:48.984521 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:48.985269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:48.985613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:48.985968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] PATH_INFO: `/v3/projects/a8d0005c494b4c6a99f2bc7fee9bd532` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:48.987222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d0005c494b4c6a99f2bc7fee9bd532)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:48.989202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:49.087774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d0005c494b4c6a99f2bc7fee9bd532', 'request_id': 'req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:49.094666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d0005c494b4c6a99f2bc7fee9bd532', 'request_id': 'req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:49.101586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d0005c494b4c6a99f2bc7fee9bd532', 'request_id': 'req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:49.172659 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] The token cache is being invalidate because project a8d0005c494b4c6a99f2bc7fee9bd532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:22:49.172956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:49.173597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:22:49.175174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1011/2057] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:22:48 2022] DELETE /identity/v3/projects/a8d0005c494b4c6a99f2bc7fee9bd532 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:22:58.629935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=25 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:59.169160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:22:59.371059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:22:59.372440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:59.383899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45313-2594-4707-b089-a4f050be4ca1 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:22:59.431887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:22:59.432403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:22:59.432676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:22:59.432939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:22:59.434614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:22:59.503926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:22:59.505345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:22:59.515969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1047/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:03.047725 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:03.049781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:03.062090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3115d5-bfac-4efc-8b9e-2c102ca3d479 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:03.066405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:03.067090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:03.067358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:03.067633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:03.069417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:03.082860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:03.084357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:03.095739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1012/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:08.935296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:08.936659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:08.950222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d008817-f3bf-4b3e-99f2-9802e01094e7 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:08.956297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:08.956889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:08.957169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:08.957449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:08.959136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:08.982852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:08.984329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:09.020318 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:09.023417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac45946-3361-463c-83aa-df7eed57dcf6 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:09.036280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1048/2060] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:09.062405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e8fea-6290-4b68-84ec-79f7fb441ab8 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:09.066479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:09.066631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:09.066806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:09.067086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:09.075628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:09.077577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:09.087004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1013/2061] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:23:09.118166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:09.118601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:09.118727 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:09.138496 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:23:09.140336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:09.174138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:23:09.197071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1049/2062] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:23:09.209959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:09.211515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:09.279840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52efa271-7c59-4c84-903f-55e8fdc47217 service glance] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:09.284830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:09.285430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:09.285710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:09.285985 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:09.294592 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:09.296253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:09.304276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1014/2063] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:23:16.952926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:16.954456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:17.009449 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38b022-1c8c-414d-9998-e8708d755da3 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:17.013509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:17.014099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:17.014372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:17.014640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:17.016352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:17.073046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:17.074476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe618d-2f06-4070-9b50-a9d7d0e703b4 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:17.083900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1050/2064] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:19.301081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=64 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:23:27.079704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=55 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:23:30.592711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:30.594112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:30.606429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c533dbd-27cd-4bdf-914a-602382481b8e service swift] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:30.610100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:30.610759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:30.611081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:30.611357 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:30.613094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:30.623914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:30.625338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:30.633531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1015/2065] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:33.117835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:33.118358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:33.118358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:33.138323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] MFA Rules not processed for user `ff0c5f8521534fc1a0a9c5ea28fbaac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:23:33.139570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:33.146355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1051/2066] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:23:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:23:39.556997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:39.570226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57f7-b69e-464b-841c-1d22c025063b service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:39.575957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:39.579324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:39.579324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:39.579324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:39.579324 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:39.591854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:39.593451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277e9df-19e5-4868-838b-5ffc7b9c18d2 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:39.604825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1016/2067] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:43.132636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:23:47.206206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:23:47.219848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052e040-424e-433e-bf1e-313619a953e0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:23:47.225218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:23:47.225810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:23:47.226096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:23:47.226371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:23:47.228093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:23:47.287786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:23:47.289368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:23:47.298506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1052/2068] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:23:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:23:49.600152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=12 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:23:57.294128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=28 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:24:05.081102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:05.081552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:05.081673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:05.101071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] MFA Rules not processed for user `e031dd9da25146eebb8813dc8bad8b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:24:05.103455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:05.109056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1017/2069] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:24:05.406384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:24:05.415990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3cc4a-f5e7-4808-86e2-87d5d2c22e3c service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:24:05.419527 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:24:05.420114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:05.420392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:05.420660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:05.422354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:05.431603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:24:05.433003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:24:05.440689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1053/2070] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:24:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:24:05.836906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:24:05.838335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:05.908243 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dea87a-3346-4de9-9cd6-a7f9f8860184 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:24:05.913082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:24:05.913697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:05.913964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:05.914235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:05.915965 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:05.927411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:24:05.928978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209156e-e45e-4477-b662-4b553f08a9d1 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:24:05.938623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1018/2071] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:24:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:24:15.437216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:24:15.934710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=35 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:24:16.547535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:24:16.549170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:16.567974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b09269-084d-47ae-bd8d-15a98d34847c service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:24:16.572586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:24:16.573320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:16.573656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:16.573996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:16.575963 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:16.597890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:24:16.597890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:24:16.601417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1054/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:24:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:24:26.597330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:24:27.654964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:27.655467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:27.655597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:27.674256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] MFA Rules not processed for user `41276c93e476436791b89ab8a0d4537d`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:24:27.676018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:27.682911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1019/2073] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:24:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:24:28.128934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:24:28.140289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a1155-0ab5-45e3-b1e1-05515d9e8bc1 service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:24:28.143932 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:24:28.144632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:28.145019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:28.145385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:28.147876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:28.159994 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:24:28.161494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89649ae1-a741-4df7-99f7-4a7f225b08d8 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:24:28.169564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1055/2074] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:24:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:24:29.341303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:24:29.354891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae61de1-9ed2-4001-8e26-008ba075dbae None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:24:29.361193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:24:29.362604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:29.362907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:29.363799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:29.364947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:29.378010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:24:29.379626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:24:29.390139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1020/2075] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:24:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:24:31.754059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:24:31.754547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:24:31.754661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:24:31.774018 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] MFA Rules not processed for user `1f737b4d0531425cbc803fcd15a3ea1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:24:31.775188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:24:31.830201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1056/2076] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:24:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:24:39.385778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=9 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:24:41.823251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=31 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:06.545711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:06.555113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ac21a-3d74-43cf-a9c7-52ee563f84c1 service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:06.558804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:06.565249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:06.565249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:06.565249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:06.565249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:06.623510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:06.624940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:06.635150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1021/2077] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:10.274282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:10.275760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:10.291102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63e6c9-86fc-421e-ac5e-4d3abdfd05d4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:10.296276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:10.296902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:10.297178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:10.297454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:10.299879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:10.311313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:10.312949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:10.323448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1057/2078] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:25:10.419385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:10.423900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:10.433199 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e3ec1-4e92-4c01-845c-90d0063d4e75 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:10.437999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:10.438634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:10.441148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:10.441148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] PATH_INFO: `/v3/users/ff0c5f8521534fc1a0a9c5ea28fbaac1` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:10.441148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0c5f8521534fc1a0a9c5ea28fbaac1)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:10.442350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:10.606325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0c5f8521534fc1a0a9c5ea28fbaac1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:10.613066 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0c5f8521534fc1a0a9c5ea28fbaac1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:10.613717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0c5f8521534fc1a0a9c5ea28fbaac1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:10.683668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0c5f8521534fc1a0a9c5ea28fbaac1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:10.684274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e51ca5db-2f72-47f1-8cc5-840b618e2c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0c5f8521534fc1a0a9c5ea28fbaac1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:10.702113 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1022/2079] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:10 2022] DELETE /identity/v3/users/ff0c5f8521534fc1a0a9c5ea28fbaac1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:10.937266 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:10.948740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fb168-8400-4d8a-b490-120efd66c929 service placement] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.005016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.005636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.005911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.006181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] PATH_INFO: `/v3/projects/9d9d2ce62fb74bdb8de53bc06b1a999d` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.007308 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d9d2ce62fb74bdb8de53bc06b1a999d)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.009193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.124262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d9d2ce62fb74bdb8de53bc06b1a999d', 'request_id': 'req-98c21b7b-2bba-450e-842b-9f31de7872d7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:11.131037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d9d2ce62fb74bdb8de53bc06b1a999d', 'request_id': 'req-98c21b7b-2bba-450e-842b-9f31de7872d7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:11.136926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d9d2ce62fb74bdb8de53bc06b1a999d', 'request_id': 'req-98c21b7b-2bba-450e-842b-9f31de7872d7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:11.197735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] The token cache is being invalidate because project 9d9d2ce62fb74bdb8de53bc06b1a999d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:11.197959 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:11.198576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-98c21b7b-2bba-450e-842b-9f31de7872d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:11.200073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1058/2080] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:10 2022] DELETE /identity/v3/projects/9d9d2ce62fb74bdb8de53bc06b1a999d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:11.205128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.205413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.205700 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.224919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:11.226102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:11.282903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1023/2081] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:11.290668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.291172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.291281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.311021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:11.312215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:11.320667 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1059/2082] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:11.324754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.334408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2da19-8f34-4cbf-899c-b66cf1edfc55 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.339203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.339809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.340092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.340366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.341014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:25:11.341322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.342973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.344428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:11.354733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1024/2083] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:25:11.358851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.368697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f4274-dc46-4566-aac6-54a99f2d2577 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.373441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.374060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.374383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.374655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.375313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.377736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.446000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1060/2084] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:11 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:11.449996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.460245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc171f-e115-4283-aaa5-3b1295629843 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.516067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.516667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.516947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.517222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.517855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.519814 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.627813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1025/2085] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:11 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:25:11.631787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.642201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed324ac-0ee8-4f1a-b037-ab7163130ece admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.646853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.647440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.647979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.648251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.652669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.652669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.660870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1061/2086] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:11.664744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.675213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fae905-cd08-424b-8b55-618c7cf745cb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.679783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.680373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.680651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.680925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.681555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.683174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae1705-76bf-4bd6-b0c4-9e58394da444 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.694124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1026/2087] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:11.698134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.708612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06a6c-6eaa-45d6-9fbb-8103220e489b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:11.713146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:11.713782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.714062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.714338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] PATH_INFO: `/v3/projects/62356c04c66f40518aa06f5bb785a0a7/users/bfe5a5d3119944d1be08966492d6174c/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.728871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62356c04c66f40518aa06f5bb785a0a7, user_id=bfe5a5d3119944d1be08966492d6174c, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:11.731581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:11.810292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1062/2088] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:11 2022] PUT /identity/v3/projects/62356c04c66f40518aa06f5bb785a0a7/users/bfe5a5d3119944d1be08966492d6174c/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:11.821973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:11.822261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:11.822559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:11.850382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] MFA Rules not processed for user `bfe5a5d3119944d1be08966492d6174c`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:11.851668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:11.917437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1027/2089] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:11.929182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:11.930886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:11.945829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:12.003327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68557b26-53c7-41d2-9c05-ff2c4c365559 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:12.009416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:12.010015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:12.014993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:12.014993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:12.018224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:12.019941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036b61-3b22-4c33-92c0-c920234ab98c service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:12.029552 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1063/2090] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:21.914583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:22.058220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=28 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:22.065895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:22.117499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4eaa5-9e68-4b3a-985d-c125073d9121 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:22.121844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:22.122457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:22.122797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:22.123145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] PATH_INFO: `/v3/users/bfe5a5d3119944d1be08966492d6174c` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:22.124217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfe5a5d3119944d1be08966492d6174c)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:22.126126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:22.257643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfe5a5d3119944d1be08966492d6174c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:22.264554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfe5a5d3119944d1be08966492d6174c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:22.265164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfe5a5d3119944d1be08966492d6174c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:22.312196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfe5a5d3119944d1be08966492d6174c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:22.312799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfe5a5d3119944d1be08966492d6174c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:22.331045 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1028/2091] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:22 2022] DELETE /identity/v3/users/bfe5a5d3119944d1be08966492d6174c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:22.335603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:22.337326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:22.337326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:22.356911 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:22.358105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:22.416139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1064/2092] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:22.425980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:22.427413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:22.484446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2565da-9cc1-4b7f-9cc4-f6cceb9fcebd admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:22.488094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:22.488688 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:22.488966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:22.489238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:22.497988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:22.499432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aa3d0-a6bd-44a4-86e3-16c3cae48e73 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:22.509151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1029/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:22.914476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:22.914476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:22.922071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18512a-82d6-48e3-97dc-7ca5502a776f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:22.927303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:22.927956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:22.928215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:22.928614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] PATH_INFO: `/v3/projects/62356c04c66f40518aa06f5bb785a0a7` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:22.929741 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62356c04c66f40518aa06f5bb785a0a7)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:22.931695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.026730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62356c04c66f40518aa06f5bb785a0a7', 'request_id': 'req-07192e8c-6fcb-4749-9005-738c9675db51'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:23.033058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62356c04c66f40518aa06f5bb785a0a7', 'request_id': 'req-07192e8c-6fcb-4749-9005-738c9675db51'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:23.039088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62356c04c66f40518aa06f5bb785a0a7', 'request_id': 'req-07192e8c-6fcb-4749-9005-738c9675db51'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:23.101622 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] The token cache is being invalidate because project 62356c04c66f40518aa06f5bb785a0a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:23.102239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:23.102457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-07192e8c-6fcb-4749-9005-738c9675db51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:23.104026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1065/2094] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:22 2022] DELETE /identity/v3/projects/62356c04c66f40518aa06f5bb785a0a7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:23.109004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.109438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.109588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.129895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:23.131124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:23.147658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:23.188589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1030/2095] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:25:23.196306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.196588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.196857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.216836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:23.218102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:23.226349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1066/2096] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:23.230438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.240955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a3aaa-bdd2-464e-a8f9-f2f8ddcb1cb2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.245488 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.246076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.246353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.246625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.247332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:25:23.247632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.249209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.260519 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1031/2097] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:25:23.264557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.273886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b2d6d-2046-400d-b8f0-a4a9d103f489 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.278482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.279103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.279380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.279646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.280302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.282192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.371409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1067/2098] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:23 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:25:23.375444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.385363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60d1cc-a60e-4161-a203-b700c3e8aa7a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.440809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.441406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.441680 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.441953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.442572 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.444675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.523813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1032/2099] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:23 2022] POST /identity/v3/users => generated 303 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:23.527964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.539053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444b10-fec9-455e-b4a0-cbf105613e5c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.543676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.544289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.544549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.544822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.545460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.547087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.557006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1068/2100] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:23.560854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.570909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e705f-6e37-47c7-8e5c-02627fd579b9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.575436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.576017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.576298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.576569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.577197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.578781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeed35-00ab-423d-a5e4-81e2b02d1793 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.589654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1033/2101] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:23.593567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.603922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c22b2-c926-450a-9468-9ab937372552 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.608471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.609096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.609371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.609640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] PATH_INFO: `/v3/projects/a0631552168d4ca6a9ddbbafcd744f83/users/2f9ca111dd474b7c8e3e2a0fe6a9ea35/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.623749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0631552168d4ca6a9ddbbafcd744f83, user_id=2f9ca111dd474b7c8e3e2a0fe6a9ea35, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.626297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.695174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1069/2102] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:23 2022] PUT /identity/v3/projects/a0631552168d4ca6a9ddbbafcd744f83/users/2f9ca111dd474b7c8e3e2a0fe6a9ea35/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:23.700149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ec29-ef39-438d-92cf-c42de88197ce None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.700438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ec29-ef39-438d-92cf-c42de88197ce None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.700714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153ec29-ef39-438d-92cf-c42de88197ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.720340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4153ec29-ef39-438d-92cf-c42de88197ce None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:23.722149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4153ec29-ef39-438d-92cf-c42de88197ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:23.780084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1034/2103] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:23.790384 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:23.792142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:23.813279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:23.848896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc93a7-7e2a-4283-8aad-8f6a4f8947b1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:23.852603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:23.853204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:23.853483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:23.853757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:23.863120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:23.864537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:23.873983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1070/2104] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:24.949288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:24.949803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:24.949909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:24.978720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] MFA Rules not processed for user `2f9ca111dd474b7c8e3e2a0fe6a9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:24.979980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:24.996631 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:25.046764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1035/2105] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:25.051962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.061769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383326a-d42a-45d0-b452-86a5538bfbae service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.067034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.067636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.067940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.068242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.068875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.070777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.142063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1071/2106] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:25 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:25.146174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.157159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3af2d-70f2-4063-8710-a095eeb08a37 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.212085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.212611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.212894 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.213167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.213813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.215753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.301516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1036/2107] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:25 2022] POST /identity/v3/users => generated 307 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:25.305636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.316051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388be051-eeec-4f78-9488-3a3fa6e4ab73 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.320709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.321298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.321577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.321850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.322521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.324163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.335818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1072/2108] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:25:25.339843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.350074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d820cf-b137-4ce2-8b95-dabe5bc165ca admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.354865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.355505 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.355780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.356051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] PATH_INFO: `/v3/projects/90fbadf96d1e4467a0a3527e147ae821/users/183dca603c7547e6ac74b8753cb2a312/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.371008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90fbadf96d1e4467a0a3527e147ae821, user_id=183dca603c7547e6ac74b8753cb2a312, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.373623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.420862 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1037/2109] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:25 2022] PUT /identity/v3/projects/90fbadf96d1e4467a0a3527e147ae821/users/183dca603c7547e6ac74b8753cb2a312/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:25:25.425476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.435559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bbe2b-e6de-46fc-97d5-94f9dec083a8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.490585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.491149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.491431 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.491703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.492364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.493936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b2674-1c59-4a6b-b308-04dd1ee8a1f5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.503429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1073/2110] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:25.507476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:25.517509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93b733-fca9-4efb-ace5-7ff63ed08811 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:25.522228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:25.522898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:25.523178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:25.523450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] PATH_INFO: `/v3/projects/90fbadf96d1e4467a0a3527e147ae821/users/183dca603c7547e6ac74b8753cb2a312/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:25.525333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90fbadf96d1e4467a0a3527e147ae821, user_id=183dca603c7547e6ac74b8753cb2a312, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:25.527988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:25.539281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1038/2111] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:25 2022] PUT /identity/v3/projects/90fbadf96d1e4467a0a3527e147ae821/users/183dca603c7547e6ac74b8753cb2a312/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:26.621271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:26.621805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:26.621969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:26.652012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] MFA Rules not processed for user `183dca603c7547e6ac74b8753cb2a312`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:26.653227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:26.721818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1074/2112] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:26 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:28.016072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:28.079826 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdabe9ef-96ea-4623-9644-77ff06cec340 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:28.084319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:28.084936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:28.085221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:28.085486 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] PATH_INFO: `/v3/users/2f9ca111dd474b7c8e3e2a0fe6a9ea35` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:28.086587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9ca111dd474b7c8e3e2a0fe6a9ea35)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:28.088559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:28.100171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:28.314498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9ca111dd474b7c8e3e2a0fe6a9ea35'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.321883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9ca111dd474b7c8e3e2a0fe6a9ea35'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.323340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9ca111dd474b7c8e3e2a0fe6a9ea35'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.398246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9ca111dd474b7c8e3e2a0fe6a9ea35'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.398905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9ca111dd474b7c8e3e2a0fe6a9ea35'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.416760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1039/2113] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:28 2022] DELETE /identity/v3/users/2f9ca111dd474b7c8e3e2a0fe6a9ea35 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:28.656044 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:28.667402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7fd70-c346-44d1-9ae4-0bf0ef695805 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:28.675342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:28.722951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:28.723585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:28.723862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:28.724131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] PATH_INFO: `/v3/projects/a0631552168d4ca6a9ddbbafcd744f83` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:28.725244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0631552168d4ca6a9ddbbafcd744f83)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:28.727202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:28.866285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0631552168d4ca6a9ddbbafcd744f83', 'request_id': 'req-36dba28c-3e9e-4e30-8bfc-c4289934bc18'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.872462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0631552168d4ca6a9ddbbafcd744f83', 'request_id': 'req-36dba28c-3e9e-4e30-8bfc-c4289934bc18'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.878010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0631552168d4ca6a9ddbbafcd744f83', 'request_id': 'req-36dba28c-3e9e-4e30-8bfc-c4289934bc18'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.945838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] The token cache is being invalidate because project a0631552168d4ca6a9ddbbafcd744f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:28.946052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.946687 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:28.948182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1075/2114] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:28 2022] DELETE /identity/v3/projects/a0631552168d4ca6a9ddbbafcd744f83 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:28.952231 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:28.953674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:28.966923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bcae8-4bd7-4e37-b8db-c180b8943a2e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:29.022398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:29.023062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.023305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.023575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] PATH_INFO: `/v3/users/183dca603c7547e6ac74b8753cb2a312` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.024671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=183dca603c7547e6ac74b8753cb2a312)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:29.026573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:29.182928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183dca603c7547e6ac74b8753cb2a312'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.189830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183dca603c7547e6ac74b8753cb2a312'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.191025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183dca603c7547e6ac74b8753cb2a312'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.255386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183dca603c7547e6ac74b8753cb2a312'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.255939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5b3a62c7-abcc-4e83-aa97-423c81d3c0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183dca603c7547e6ac74b8753cb2a312'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.274426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1040/2115] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:28 2022] DELETE /identity/v3/users/183dca603c7547e6ac74b8753cb2a312 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:29.479207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:29.480701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:29.494125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dba28c-3e9e-4e30-8bfc-c4289934bc18 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:29.551059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:29.551689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.551968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.552234 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] PATH_INFO: `/v3/projects/90fbadf96d1e4467a0a3527e147ae821` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.553316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] RBAC: Authorizing `identity:delete_project(project_id=90fbadf96d1e4467a0a3527e147ae821)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:29.555247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:29.645358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:29.649179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90fbadf96d1e4467a0a3527e147ae821', 'request_id': 'req-15f84283-71b6-4b8b-b7f0-6e709b626def'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.655273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90fbadf96d1e4467a0a3527e147ae821', 'request_id': 'req-15f84283-71b6-4b8b-b7f0-6e709b626def'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.660689 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90fbadf96d1e4467a0a3527e147ae821', 'request_id': 'req-15f84283-71b6-4b8b-b7f0-6e709b626def'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.711720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] The token cache is being invalidate because project 90fbadf96d1e4467a0a3527e147ae821 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:29.712041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.712674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15f84283-71b6-4b8b-b7f0-6e709b626def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:29.714229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1076/2116] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:29 2022] DELETE /identity/v3/projects/90fbadf96d1e4467a0a3527e147ae821 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:29.719131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.719743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.719743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.739664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:29.740929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:29.745336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:29.800921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1041/2117] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:29.808697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.808983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.809255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.829062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:29.830241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:29.838864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1077/2118] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:29.843008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:29.852452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a40e-c5d2-4ec2-92c2-9210dc1e14e4 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:29.856942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:29.857530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.857804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.858075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.858710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:25:29.859040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:29.860645 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:29.871755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1042/2119] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:25:29.875811 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:29.885467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1146f84-6572-44a2-8b2f-276397612f97 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:29.890047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:29.890659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:29.890975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:29.891252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:29.891883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:29.894421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:29.987542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1078/2120] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:29 2022] POST /identity/v3/projects => generated 353 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:25:29.991692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.001449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9710f5-ad55-46e0-8e59-6ce225cf04f7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.056388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.057014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.057166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.057433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.058040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.059968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.147495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1043/2121] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:29 2022] POST /identity/v3/users => generated 277 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:30.151683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.161949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c516e-66e3-479b-8d73-26efba16f713 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.166541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.167163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.167473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.167748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.168386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.169973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.180177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1079/2122] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:30.184027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.194596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d618dfd-d4ba-4781-82a2-e739b5d67bf0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.199207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.199796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.200077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.200351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.201002 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.202588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b51816-7623-4776-b1d8-5ea3424525ca admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.213026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1044/2123] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:30.216907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.226635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945e8c6-dff5-4646-9b37-c957998390bf admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.231169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.231796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.232067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.232336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] PATH_INFO: `/v3/projects/de36ffdefe4d4f08ad5cac8cea3f0448/users/c93c90145d93497eb1bbb8a7fbd6bae8/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.247005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de36ffdefe4d4f08ad5cac8cea3f0448, user_id=c93c90145d93497eb1bbb8a7fbd6bae8, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.249614 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.318613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1080/2124] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:30 2022] PUT /identity/v3/projects/de36ffdefe4d4f08ad5cac8cea3f0448/users/c93c90145d93497eb1bbb8a7fbd6bae8/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:30.332350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.332637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.332939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.360124 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] MFA Rules not processed for user `c93c90145d93497eb1bbb8a7fbd6bae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:30.361326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:30.423377 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1045/2125] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:30.428343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.487228 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a523e-3c9e-4cf6-82bd-596566f44660 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.492238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.492822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.493098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.493365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.493973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.495879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.587357 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1081/2126] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:30 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:30.591530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.598793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:30.603042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6a3d8-1d75-4b85-8046-42f9f971e46c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.659564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.660656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.660939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.661217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.661837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.663804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.720509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1046/2127] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:30 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:30.721960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.732862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44da5b-b1ed-43dd-afc3-6d7444828f48 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.737451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.738028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.738973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.739241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.739878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.743248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.753330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1082/2128] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:30.757632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.767910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccccd3-cf11-4fcf-88b3-b61a9bf24cd7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.772808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.773451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.773730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.774003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] PATH_INFO: `/v3/projects/adc9a8d2bd644485831945dc66198829/users/46a3b3b9b9fd4987aa53ddc38b3debdf/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.788927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc9a8d2bd644485831945dc66198829, user_id=46a3b3b9b9fd4987aa53ddc38b3debdf, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.791614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.860833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1047/2129] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:30 2022] PUT /identity/v3/projects/adc9a8d2bd644485831945dc66198829/users/46a3b3b9b9fd4987aa53ddc38b3debdf/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:30.865497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.875486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55706503-49d9-4954-b648-3c0c9796ae99 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.914495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:30.930884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.931479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.931753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.932022 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.932647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.934412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789bd79-e0ff-4661-88ca-e4c9317ec119 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.943922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1083/2130] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:30.948048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:30.958112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa618ce-c1c5-4785-a09a-3e0302a7228f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:30.962799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:30.963472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.963752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.964020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] PATH_INFO: `/v3/projects/adc9a8d2bd644485831945dc66198829/users/46a3b3b9b9fd4987aa53ddc38b3debdf/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:30.965967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc9a8d2bd644485831945dc66198829, user_id=46a3b3b9b9fd4987aa53ddc38b3debdf, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:30.968661 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:30.979193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1048/2131] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:30 2022] PUT /identity/v3/projects/adc9a8d2bd644485831945dc66198829/users/46a3b3b9b9fd4987aa53ddc38b3debdf/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:30.994299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:30.994577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:30.994876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:31.022149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] MFA Rules not processed for user `46a3b3b9b9fd4987aa53ddc38b3debdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:31.023390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:31.088758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1084/2132] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:31.100198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:31.101959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:31.159320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1bf71-e409-4c34-a51b-ac93cd6d322e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:31.163387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:31.163998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:31.164262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:31.164533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:31.173269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:31.174684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:31.184473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1049/2133] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:31.378726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:31.388892 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c39f4c-59aa-4046-bee3-e2336ec5252c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:31.392949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:31.393556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:31.393838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:31.394112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:31.452809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:31.454150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f860446-f7d0-41cc-a9f4-c3f96ec813a2 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:31.463925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1085/2134] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:41.180849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:41.459810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=72 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:50.955169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:51.016812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc2334-d8f2-4b06-b8b3-c82cc291c6ad service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:51.021273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:51.022150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:51.022451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:51.022722 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] PATH_INFO: `/v3/users/c93c90145d93497eb1bbb8a7fbd6bae8` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:51.023853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93c90145d93497eb1bbb8a7fbd6bae8)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:51.025783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:51.222551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93c90145d93497eb1bbb8a7fbd6bae8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:51.229934 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93c90145d93497eb1bbb8a7fbd6bae8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:51.230600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93c90145d93497eb1bbb8a7fbd6bae8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:51.300713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93c90145d93497eb1bbb8a7fbd6bae8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:51.301338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93c90145d93497eb1bbb8a7fbd6bae8'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:51.320264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1050/2135] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:50 2022] DELETE /identity/v3/users/c93c90145d93497eb1bbb8a7fbd6bae8 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:51.324700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:51.325107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:51.325338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:51.345601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:51.346894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:51.407670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1086/2136] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:51.417993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:51.419425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:51.476274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bd294-e4dc-4922-8772-d3b542463b42 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:51.480618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:51.481257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:51.481538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:51.481812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:51.490742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:51.492202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:51.501689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1051/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:25:51.897163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:51.898561 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:51.911063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08e99ae-e55e-4900-b3f4-d926a06c3471 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:51.915931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:51.923411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:51.923411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:51.923411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] PATH_INFO: `/v3/projects/de36ffdefe4d4f08ad5cac8cea3f0448` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:51.923411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] RBAC: Authorizing `identity:delete_project(project_id=de36ffdefe4d4f08ad5cac8cea3f0448)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:51.923411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:52.080203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de36ffdefe4d4f08ad5cac8cea3f0448', 'request_id': 'req-b059a581-7bb3-454f-b438-2fe848f5c86d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.086549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de36ffdefe4d4f08ad5cac8cea3f0448', 'request_id': 'req-b059a581-7bb3-454f-b438-2fe848f5c86d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.099804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de36ffdefe4d4f08ad5cac8cea3f0448', 'request_id': 'req-b059a581-7bb3-454f-b438-2fe848f5c86d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.163414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] The token cache is being invalidate because project de36ffdefe4d4f08ad5cac8cea3f0448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:52.163706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.164399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.166085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1087/2138] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:51 2022] DELETE /identity/v3/projects/de36ffdefe4d4f08ad5cac8cea3f0448 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:52.170882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:52.181039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a263c-144d-46dd-9326-be026f303d75 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:52.200029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:52.238896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:52.241914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:52.241914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:52.241914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] PATH_INFO: `/v3/users/46a3b3b9b9fd4987aa53ddc38b3debdf` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:52.241914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a3b3b9b9fd4987aa53ddc38b3debdf)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:52.243063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:52.440006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a3b3b9b9fd4987aa53ddc38b3debdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.446923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a3b3b9b9fd4987aa53ddc38b3debdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.447576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a3b3b9b9fd4987aa53ddc38b3debdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.517287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a3b3b9b9fd4987aa53ddc38b3debdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.517921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2eb96279-17fd-4c73-aa64-11469c568166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a3b3b9b9fd4987aa53ddc38b3debdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:52.535831 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1052/2139] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:25:52 2022] DELETE /identity/v3/users/46a3b3b9b9fd4987aa53ddc38b3debdf => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:52.971353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:52.972924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:52.988002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059a581-7bb3-454f-b438-2fe848f5c86d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.045399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.046037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.046337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.046620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] PATH_INFO: `/v3/projects/adc9a8d2bd644485831945dc66198829` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.047763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc9a8d2bd644485831945dc66198829)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.049660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.141930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:53.163057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc9a8d2bd644485831945dc66198829', 'request_id': 'req-15835836-c6ef-4c5f-a60c-e0924e1defac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:53.168805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc9a8d2bd644485831945dc66198829', 'request_id': 'req-15835836-c6ef-4c5f-a60c-e0924e1defac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:53.174202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc9a8d2bd644485831945dc66198829', 'request_id': 'req-15835836-c6ef-4c5f-a60c-e0924e1defac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:53.242513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] The token cache is being invalidate because project adc9a8d2bd644485831945dc66198829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:25:53.242733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:53.243373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-15835836-c6ef-4c5f-a60c-e0924e1defac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:25:53.244901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1088/2140] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:25:52 2022] DELETE /identity/v3/projects/adc9a8d2bd644485831945dc66198829 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:53.250450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.250947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.251071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.270709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:53.272184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:53.330191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1053/2141] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:53.337839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.338123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.338395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.358198 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:53.359436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:53.367323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1089/2142] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:53.371991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.381397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676cd8-e31b-4b80-a7b8-6e3d428f9d2d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.386396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.387020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.387303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.387576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.388216 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:25:53.388518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.390105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.402413 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1054/2143] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:25:53.406432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.415493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c9d3-9d4c-421d-bdc5-e5123833cbe4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.420573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.421150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.421426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.421693 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.422316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.424227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.524342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1090/2144] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:53.528967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.538959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1cc29-8852-4f92-83fb-c22e00b19566 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.564784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:53.595689 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.596289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.596608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.597101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.598229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.600218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.701288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1055/2145] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:53.705343 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.715739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73938f0-bd72-4308-949e-a69f0517017e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.720424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.721032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.721338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.721613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.722255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.723931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.734270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1091/2146] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:53.738174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.748749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5464248-70bb-43b9-a318-2434c50ce309 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.753532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.754303 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.754581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.754878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.755518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.757097 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978d87f-7a67-42ee-ae4f-63646b1ecb23 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.768532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1056/2147] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:53.772898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:53.782799 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180beaff-acd0-4041-8ecc-697f7deb3fb8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:53.787328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:53.787964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.788244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.788518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] PATH_INFO: `/v3/projects/689f5bdffba1443884ddab058e46a8c6/users/79b6da2caed446da84a1445ae38b41e9/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.803919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=689f5bdffba1443884ddab058e46a8c6, user_id=79b6da2caed446da84a1445ae38b41e9, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:53.806547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:53.877903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1092/2148] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:53 2022] PUT /identity/v3/projects/689f5bdffba1443884ddab058e46a8c6/users/79b6da2caed446da84a1445ae38b41e9/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:53.888671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:53.888957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:53.889232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:53.916906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] MFA Rules not processed for user `79b6da2caed446da84a1445ae38b41e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:53.918120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:53.983163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1057/2149] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:53.987711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.047885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec64b79-3c62-4444-bcd3-87978e3b9cca admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.052426 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.053021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.053301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.053573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.054192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.056111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.152720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1093/2150] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:25:53 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:54.156795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.168400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2f212-7b47-48fd-acc7-2af71359e8ea None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.225578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.226181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.226461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.226735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.227393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.229791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.346214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1058/2151] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:25:54 2022] POST /identity/v3/users => generated 295 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:25:54.350059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.360733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef401f-0083-4783-84d5-105d8fb4fe1e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.365458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.366047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.366341 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.366597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.367271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.368931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.379209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1094/2152] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:54.383341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.393224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7775bde-ddaf-4cc6-8e6e-f9b0f469cead admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.398121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.398762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.399076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.399375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] PATH_INFO: `/v3/projects/149f0a58bdc9494894293187270420bb/users/84f0fbe4f9b24d8cb1345536ea439ddb/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.414335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=149f0a58bdc9494894293187270420bb, user_id=84f0fbe4f9b24d8cb1345536ea439ddb, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.417669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.483660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1059/2153] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:54 2022] PUT /identity/v3/projects/149f0a58bdc9494894293187270420bb/users/84f0fbe4f9b24d8cb1345536ea439ddb/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:54.487551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.497573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7c677-d5fe-4f7c-8259-8515491abbad admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.517208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:54.553510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.554107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.554387 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.554663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.555328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.556957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018f3e1-639b-4f22-a58c-dc94a5ad2209 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.566869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1095/2154] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:25:54.571003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.577537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:25:54.582789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bc53d-7736-47f8-913d-528d5b5f0a6c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.587579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.588220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.588499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.589269 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] PATH_INFO: `/v3/projects/149f0a58bdc9494894293187270420bb/users/84f0fbe4f9b24d8cb1345536ea439ddb/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.591198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149f0a58bdc9494894293187270420bb, user_id=84f0fbe4f9b24d8cb1345536ea439ddb, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.597558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.603987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1060/2155] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:25:54 2022] PUT /identity/v3/projects/149f0a58bdc9494894293187270420bb/users/84f0fbe4f9b24d8cb1345536ea439ddb/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:25:54.614755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.615065 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.615340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.642210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] MFA Rules not processed for user `84f0fbe4f9b24d8cb1345536ea439ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:25:54.644397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:54.711506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1096/2156] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:25:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:25:54.725634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:25:54.727453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:25:54.784517 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46688f35-c795-49c3-ba47-a919f5b8d1c9 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:25:54.788125 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:25:54.788718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:25:54.788994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:25:54.789262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:25:54.844340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:25:54.845764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:25:54.854565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1061/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:25:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:03.256476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:03.258092 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:03.300211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae74e8e-868f-4432-9b73-d86580298896 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:03.304008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:03.304621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:03.304904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:03.305176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:03.335554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:03.336970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a915cbf-920e-4054-9af2-5aff1bbdaaab service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:03.347207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1097/2158] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:26:04.850471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:13.342886 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=80 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:17.596473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:17.598110 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:17.737015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc558cd-36b2-42c8-a9fc-1d3f35c2ed74 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:17.747556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:17.748891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:17.749178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:17.749938 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:17.753371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:17.766341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:17.768004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:17.778221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1062/2159] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:22.280285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:22.280805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:22.280805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:22.299572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] MFA Rules not processed for user `e031dd9da25146eebb8813dc8bad8b85`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:22.300893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:22.354143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1098/2160] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:24.899238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:24.900716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:24.975799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d61f1-5f6f-4ea0-b8ce-702b5fccef45 service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:24.979326 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:24.979919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:24.980195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:24.980462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:24.982765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:25.023675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:25.041669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:25.043178 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af72e0-b37f-4d5c-b89d-5b568fdd007f service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:25.052794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1063/2161] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:32.347594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=25 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:35.100972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=9 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:36.719649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:36.800549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff4a92-8249-4a51-b4f2-bc5de2ee3296 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:36.805316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:36.805946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:36.806226 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:36.806500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] PATH_INFO: `/v3/users/79b6da2caed446da84a1445ae38b41e9` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:36.807651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b6da2caed446da84a1445ae38b41e9)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:36.809621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:37.016350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b6da2caed446da84a1445ae38b41e9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.024632 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b6da2caed446da84a1445ae38b41e9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.025774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b6da2caed446da84a1445ae38b41e9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.083903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b6da2caed446da84a1445ae38b41e9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.084649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b6da2caed446da84a1445ae38b41e9'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.104224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1099/2162] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:36 2022] DELETE /identity/v3/users/79b6da2caed446da84a1445ae38b41e9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:37.108628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:37.109186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:37.109186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:37.152852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:37.154803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:37.225167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1064/2163] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:37.235592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:37.237068 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:37.302001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bfc832-89ac-4093-a7df-3bf1c8db9985 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:37.305855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:37.306469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:37.306751 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:37.307058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:37.317858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:37.319349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:37.335366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1100/2164] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:37.725918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:37.727761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:37.749271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c5261-eaf9-4c18-ace7-406d45f5b434 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:37.754000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:37.754685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:37.755906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:37.755906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] PATH_INFO: `/v3/projects/689f5bdffba1443884ddab058e46a8c6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:37.756556 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689f5bdffba1443884ddab058e46a8c6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:37.758636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:37.901807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689f5bdffba1443884ddab058e46a8c6', 'request_id': 'req-10699a75-baa8-4549-ba66-53d43cd454a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.908663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689f5bdffba1443884ddab058e46a8c6', 'request_id': 'req-10699a75-baa8-4549-ba66-53d43cd454a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:37.914261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689f5bdffba1443884ddab058e46a8c6', 'request_id': 'req-10699a75-baa8-4549-ba66-53d43cd454a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.009132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] The token cache is being invalidate because project 689f5bdffba1443884ddab058e46a8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:38.009407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.010033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.011563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1065/2165] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:37 2022] DELETE /identity/v3/projects/689f5bdffba1443884ddab058e46a8c6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:38.016135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:38.022805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:38.026351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608bfafd-08e1-4d6c-b052-bc94a2bb1ffb service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:38.081100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:38.081720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:38.081999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:38.082273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] PATH_INFO: `/v3/users/84f0fbe4f9b24d8cb1345536ea439ddb` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:38.083405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f0fbe4f9b24d8cb1345536ea439ddb)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:38.085306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:38.314730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f0fbe4f9b24d8cb1345536ea439ddb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.321624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f0fbe4f9b24d8cb1345536ea439ddb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.322277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f0fbe4f9b24d8cb1345536ea439ddb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.398271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f0fbe4f9b24d8cb1345536ea439ddb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.398982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f0fbe4f9b24d8cb1345536ea439ddb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.417139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1101/2166] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:38 2022] DELETE /identity/v3/users/84f0fbe4f9b24d8cb1345536ea439ddb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:38.718131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:38.721453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:38.778795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10699a75-baa8-4549-ba66-53d43cd454a1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:38.797694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:38.805026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:38.806486 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:38.836635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:38.837206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:38.837487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:38.837759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:38.839839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:38.873303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f84ac-55ed-4ef0-b3ef-87d720870374 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:38.878024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:38.878655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:38.878966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:38.879243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] PATH_INFO: `/v3/projects/149f0a58bdc9494894293187270420bb` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:38.880362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=149f0a58bdc9494894293187270420bb)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:38.882504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:38.905812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:38.907284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22581da-6f83-4cbf-8522-5942020a3557 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:38.917160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1066/2167] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:38.971202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:38.987453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149f0a58bdc9494894293187270420bb', 'request_id': 'req-e285d861-d97b-4156-8698-c2b64ebbab5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.993913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149f0a58bdc9494894293187270420bb', 'request_id': 'req-e285d861-d97b-4156-8698-c2b64ebbab5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:38.999396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149f0a58bdc9494894293187270420bb', 'request_id': 'req-e285d861-d97b-4156-8698-c2b64ebbab5e'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:39.089806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] The token cache is being invalidate because project 149f0a58bdc9494894293187270420bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:39.090482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:39.091005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e285d861-d97b-4156-8698-c2b64ebbab5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:39.093169 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1102/2168] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:38 2022] DELETE /identity/v3/projects/149f0a58bdc9494894293187270420bb => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:26:39.098210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.098599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.099013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.149197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:39.150446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:39.220885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1067/2169] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:39.228910 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.229144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.229584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.252219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:39.253559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:39.262016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1103/2170] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:39.267220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.287771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5722b2-bb9f-441a-900f-f4b4b7f9ddee None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.294917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.297010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.297010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.297010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.297010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:26:39.297384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.299053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.313190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1068/2171] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:26:39.317325 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.329722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05618c3e-cffc-4380-8f7b-e47de5269690 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.334858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.335498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.335742 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.336009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.336638 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.338623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.449783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1104/2172] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:26:39.450971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.461034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad03c4d-1e93-4bb2-b2ba-f2275f8a7d7d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.496603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:39.524538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.525241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.525588 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.525925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.526630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.528812 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.648739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1069/2173] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:26:39.649966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.661435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec96cd-d64d-4ca6-9bae-745cd641cabe admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.666577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.667243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.667526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.667990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.668658 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.670300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.681713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1105/2174] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:26:39.685970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.697703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f29ede-7df0-49e3-b1cb-ac8a942b22fb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.703462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.704162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.704510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.704874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.705635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.707451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa8070-506f-4653-a191-6c9275ee33b7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.720778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1070/2175] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:26:39.725292 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:39.737015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39198e-252c-472f-81f0-72c8daee80b3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:39.743391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:39.744223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.744517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.744788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] PATH_INFO: `/v3/projects/1e58929a8c8b43f59072ba94526ca504/users/53a3f5f3f9d941dd9ad34361fca5d2eb/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.762796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e58929a8c8b43f59072ba94526ca504, user_id=53a3f5f3f9d941dd9ad34361fca5d2eb, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:39.766041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:39.850321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1106/2176] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:39 2022] PUT /identity/v3/projects/1e58929a8c8b43f59072ba94526ca504/users/53a3f5f3f9d941dd9ad34361fca5d2eb/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:39.860775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:39.861060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:39.861338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:39.890684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] MFA Rules not processed for user `53a3f5f3f9d941dd9ad34361fca5d2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:39.891942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:39.957787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1071/2177] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:39.962105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.025380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fdbf8-71e2-4afb-9a2f-910e3ac44ea6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.029973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.034686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.034686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.034686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.034686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.034686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.124396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1107/2178] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:26:39 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:26:40.128496 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.141448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f255e7f7-c5a8-4fb6-aef3-fff76822bcaf None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.206121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.206743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.207064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.207335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.208046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.209923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.305017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1072/2179] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:26:40 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:26:40.308570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.322296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d085a00-d058-46c3-b4a1-884c87f217e8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.328041 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.328621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.328904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.329177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.329969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.332358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.347130 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1108/2180] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:26:40.348381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.359348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dbeeb-7ecb-4b17-a0c0-19c085d11fb3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.364158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.364924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.365279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.365624 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] PATH_INFO: `/v3/projects/2f1379987a004f748b779da5eaac8677/users/79240c44ffdf4f46ac57363010c7b944/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.385226 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1379987a004f748b779da5eaac8677, user_id=79240c44ffdf4f46ac57363010c7b944, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.388065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.480859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1073/2181] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:40 2022] PUT /identity/v3/projects/2f1379987a004f748b779da5eaac8677/users/79240c44ffdf4f46ac57363010c7b944/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:26:40.482084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.495761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f9a98-54b8-4d47-9c58-b3186ba3eb84 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.518435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:40.564922 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.565608 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.565890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.566166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.566844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.568462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f29cb-4001-4fb0-8d61-3db670d19966 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.581870 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1109/2182] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:26:40.583128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.596248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7fb7-efa9-4f2f-8ab4-48cd82074304 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.602634 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.603440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.606943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.606943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] PATH_INFO: `/v3/projects/2f1379987a004f748b779da5eaac8677/users/79240c44ffdf4f46ac57363010c7b944/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.620941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1379987a004f748b779da5eaac8677, user_id=79240c44ffdf4f46ac57363010c7b944, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:40.624494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:40.628273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:40.638750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1074/2183] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:40 2022] PUT /identity/v3/projects/2f1379987a004f748b779da5eaac8677/users/79240c44ffdf4f46ac57363010c7b944/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:40.712529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.712821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.713074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.762376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] MFA Rules not processed for user `79240c44ffdf4f46ac57363010c7b944`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:40.763694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:40.858861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1110/2184] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:40.867996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:40.870078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:40.981561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa16491-1fd2-448a-af08-664a4fd775a6 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:40.986608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:40.987291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:40.987574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:40.987847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:40.997830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:41.001353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785487fd-f7cd-4927-9350-5361d48bfced service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:41.010277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1075/2185] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:41.094420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:41.173213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5956e-9aae-4d04-b0b6-b2110dd6d6d4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:41.180276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:41.181143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:41.181578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:41.182002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] PATH_INFO: `/v3/users/53a3f5f3f9d941dd9ad34361fca5d2eb` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:41.183375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a3f5f3f9d941dd9ad34361fca5d2eb)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:41.186321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:41.403620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a3f5f3f9d941dd9ad34361fca5d2eb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:41.405393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:41.411914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a3f5f3f9d941dd9ad34361fca5d2eb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:41.412628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a3f5f3f9d941dd9ad34361fca5d2eb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:41.483931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a3f5f3f9d941dd9ad34361fca5d2eb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:41.484497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a3f5f3f9d941dd9ad34361fca5d2eb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:41.502831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1111/2186] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:41 2022] DELETE /identity/v3/users/53a3f5f3f9d941dd9ad34361fca5d2eb => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:41.507330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:41.507614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:41.507884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:41.529490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:41.534294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:41.593968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1076/2187] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:41.603940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:41.606691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:41.700080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed205d6-290a-49ae-8184-c93c2901f3d8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:41.704938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:41.706475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:41.707242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:41.707695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:41.724315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:41.726103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:41.737894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1112/2188] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:42.405555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:42.407190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:42.432028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75453d40-2a0e-440a-99b1-c30f22f9abf7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:42.436382 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:42.437039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:42.437323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:42.437598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] PATH_INFO: `/v3/projects/1e58929a8c8b43f59072ba94526ca504` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:42.441048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e58929a8c8b43f59072ba94526ca504)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:42.448341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:42.612379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e58929a8c8b43f59072ba94526ca504', 'request_id': 'req-6b724ccb-f092-44ba-87b4-9bffd4258679'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:42.617096 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e58929a8c8b43f59072ba94526ca504', 'request_id': 'req-6b724ccb-f092-44ba-87b4-9bffd4258679'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:42.624381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e58929a8c8b43f59072ba94526ca504', 'request_id': 'req-6b724ccb-f092-44ba-87b4-9bffd4258679'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:42.730727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] The token cache is being invalidate because project 1e58929a8c8b43f59072ba94526ca504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:42.730951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:42.731581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:42.733256 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1077/2189] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:42 2022] DELETE /identity/v3/projects/1e58929a8c8b43f59072ba94526ca504 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:42.737414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:42.747777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afa5c6-4dc4-4e24-a933-ef81c1968d96 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:42.805396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:42.806012 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:42.806295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:42.806789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] PATH_INFO: `/v3/users/79240c44ffdf4f46ac57363010c7b944` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:42.807837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=79240c44ffdf4f46ac57363010c7b944)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:42.810061 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:42.989283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:42.993235 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:42.999278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:43.020498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79240c44ffdf4f46ac57363010c7b944'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:43.033172 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79240c44ffdf4f46ac57363010c7b944'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:43.034466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79240c44ffdf4f46ac57363010c7b944'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:43.075558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724ccb-f092-44ba-87b4-9bffd4258679 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:43.082171 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:43.083390 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:43.083780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:43.084117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:43.085846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:43.097970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:43.099895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:43.111632 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1078/2190] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:26:43.135662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79240c44ffdf4f46ac57363010c7b944'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:43.137392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-38c5c052-b7b5-4184-9f2f-c17ffd4c4bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79240c44ffdf4f46ac57363010c7b944'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:43.144572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:43.158923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1113/2191] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:42 2022] DELETE /identity/v3/users/79240c44ffdf4f46ac57363010c7b944 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:43.968364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:43.969964 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:43.991909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569ca28-2c56-4c02-9572-2060775e9815 service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:44.116418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:44.117660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.118161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.118726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] PATH_INFO: `/v3/projects/2f1379987a004f748b779da5eaac8677` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.120250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1379987a004f748b779da5eaac8677)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:44.122791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:44.257792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1379987a004f748b779da5eaac8677', 'request_id': 'req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:44.267752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1379987a004f748b779da5eaac8677', 'request_id': 'req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:44.275232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1379987a004f748b779da5eaac8677', 'request_id': 'req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:44.388219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] The token cache is being invalidate because project 2f1379987a004f748b779da5eaac8677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:44.388690 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:44.390465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-ea1c9b55-9b8e-456b-90c2-3bc9d2e91e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:44.392578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1079/2192] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:43 2022] DELETE /identity/v3/projects/2f1379987a004f748b779da5eaac8677 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:44.397760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.398244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.398346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.423844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:44.425395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:44.540761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1114/2193] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:26:44.548864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.549446 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.549981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.573208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:44.574520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:44.593782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:44.599356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1080/2194] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:44.610491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f2ccf-92b8-4cf9-b62b-950169561ee5 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:44.621412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:44.622210 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.622590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.623003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.624510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:26:44.624916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:44.626685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:44.647795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1115/2195] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:26:44.648583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:44.659236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70e1b3-bd39-456b-b908-ab384df548de None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:44.667952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:44.668571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.668847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.669240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.669872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:44.671884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:44.689827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:44.794714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1081/2196] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:26:44 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:26:44.794960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:44.807969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193d395-0701-453d-a219-4941e871e13e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:44.879968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:44.880995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:44.881324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:44.881671 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:44.882467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:44.885060 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:45.002528 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:45.017814 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1116/2197] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:26:44 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:26:45.020261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7e73a-e35c-454b-8d4f-96c605aa5c91 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:45.031049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:45.031847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:45.032236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:45.032612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:45.033358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:45.035748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:45.053189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1082/2198] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:26:45.053409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:45.070792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5aef3-a2fd-4fce-9e54-b8350766004c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:45.083275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:45.087140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:45.087664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:45.088600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:45.089785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:45.091804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1a144-3392-480f-b3bb-7055c144ba7a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:45.119676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:45.124306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1117/2199] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:26:45.132989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ee02-ae47-4e25-925e-dd22377013fc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:45.140057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:45.141044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:45.141481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:45.141977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] PATH_INFO: `/v3/projects/bc09774cd4634db792679d5276ea6a27/users/e4d8414cfc954d2d81505e863b670096/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:45.160768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc09774cd4634db792679d5276ea6a27, user_id=e4d8414cfc954d2d81505e863b670096, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:45.164451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:45.235333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1083/2200] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:45 2022] PUT /identity/v3/projects/bc09774cd4634db792679d5276ea6a27/users/e4d8414cfc954d2d81505e863b670096/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:45.240997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8674-6170-4fcc-9c44-cb34f1e25915 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:45.241518 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8674-6170-4fcc-9c44-cb34f1e25915 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:45.241957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a8674-6170-4fcc-9c44-cb34f1e25915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:45.265719 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-451a8674-6170-4fcc-9c44-cb34f1e25915 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:45.266680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-451a8674-6170-4fcc-9c44-cb34f1e25915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:45.342645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:45.365866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1118/2201] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:45.371514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:45.373472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:45.466142 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9505-c673-493f-ab58-264519d39155 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:45.470972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:45.471731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:45.472111 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:45.472586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:45.487714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:45.489283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:45.505094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1084/2202] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:47.037688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.038239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.038239 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.087498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] MFA Rules not processed for user `e4d8414cfc954d2d81505e863b670096`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:47.088941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:47.180530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1119/2203] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:47.185071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.196840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07af71-1980-4ba4-9d10-376df4437cd4 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.201411 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.202024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.202299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.202573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.203248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.205133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.320942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1085/2204] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:26:47 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:26:47.325688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.342663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0db115-2c92-45b4-9f04-fadb5df97367 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.405133 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.409099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.409099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.409099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.409099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.409648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.491537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1120/2205] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:26:47 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:26:47.495482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.508639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989ff57-3fa3-4c4f-af35-27547e19e373 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.513958 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.514620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.514857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.515267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.515851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.517483 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.530621 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1086/2206] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:26:47.535110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.546931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b905f42-404e-4a8e-80c8-a312ef80b3a9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.553542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.554207 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.554485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.554868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] PATH_INFO: `/v3/projects/6453923e032b4d9185288d458faed4d7/users/049cb2fda8b74469945ac9aa7ae218cb/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.570276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6453923e032b4d9185288d458faed4d7, user_id=049cb2fda8b74469945ac9aa7ae218cb, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.573100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.634059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.635827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:47.649866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d953a-337e-4d91-9374-fee9600df2c5 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.657860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1121/2207] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:47 2022] PUT /identity/v3/projects/6453923e032b4d9185288d458faed4d7/users/049cb2fda8b74469945ac9aa7ae218cb/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:26:47.661966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.722106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.722767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.723398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.723673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] PATH_INFO: `/v3/users/ce09c776cd77416091ea214b858afacd` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.724665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:47.725015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce09c776cd77416091ea214b858afacd)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.726953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.745897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15c9d2-bad2-436b-85ff-16182e0dd13d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.749862 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.750481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.750852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.751117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.751749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.753339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.763933 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1122/2208] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:26:47.768078 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:47.778071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5775-91be-4d1a-896f-e2cd30eb0959 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:47.783347 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:47.783988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:47.784264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:47.784535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] PATH_INFO: `/v3/projects/6453923e032b4d9185288d458faed4d7/users/049cb2fda8b74469945ac9aa7ae218cb/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:47.786448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6453923e032b4d9185288d458faed4d7, user_id=049cb2fda8b74469945ac9aa7ae218cb, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:47.789110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:47.800011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1123/2209] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:26:47 2022] PUT /identity/v3/projects/6453923e032b4d9185288d458faed4d7/users/049cb2fda8b74469945ac9aa7ae218cb/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:26:47.878373 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:47.893392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce09c776cd77416091ea214b858afacd'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:47.901186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce09c776cd77416091ea214b858afacd'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:47.901946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce09c776cd77416091ea214b858afacd'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:47.993331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce09c776cd77416091ea214b858afacd'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:47.993972 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce09c776cd77416091ea214b858afacd'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.012668 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1087/2210] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:47 2022] DELETE /identity/v3/users/ce09c776cd77416091ea214b858afacd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:48.347836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:48.349589 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:48.437030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833ab80-9b6c-4cb7-a865-1997a867eeea admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:48.503905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:48.504537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:48.504813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:48.505085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] PATH_INFO: `/v3/projects/a30976e8b35f436b84dc4488dcf5acaf` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:48.506181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30976e8b35f436b84dc4488dcf5acaf)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:48.508107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:48.611618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30976e8b35f436b84dc4488dcf5acaf', 'request_id': 'req-7559a890-42db-4543-a10e-c180b7e3c17c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.618218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30976e8b35f436b84dc4488dcf5acaf', 'request_id': 'req-7559a890-42db-4543-a10e-c180b7e3c17c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.625425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30976e8b35f436b84dc4488dcf5acaf', 'request_id': 'req-7559a890-42db-4543-a10e-c180b7e3c17c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.731935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] The token cache is being invalidate because project a30976e8b35f436b84dc4488dcf5acaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:48.732350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.733615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7559a890-42db-4543-a10e-c180b7e3c17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.736155 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1124/2211] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:48 2022] DELETE /identity/v3/projects/a30976e8b35f436b84dc4488dcf5acaf => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:48.740394 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:48.755909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fb1f2-976b-4afa-9974-9603ffde4dee admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:48.811436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:48.812069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:48.812344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:48.812617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] PATH_INFO: `/v3/users/aea36f8f711b499dab1e28931d87a07a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:48.813718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aea36f8f711b499dab1e28931d87a07a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:48.815795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:48.990508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aea36f8f711b499dab1e28931d87a07a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.997677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aea36f8f711b499dab1e28931d87a07a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:48.998338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aea36f8f711b499dab1e28931d87a07a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.044011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aea36f8f711b499dab1e28931d87a07a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.044663 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aea36f8f711b499dab1e28931d87a07a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.067091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1088/2212] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:26:48 2022] DELETE /identity/v3/users/aea36f8f711b499dab1e28931d87a07a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:49.116454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:49.116903 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:49.117257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:49.145851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:49.160718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] MFA Rules not processed for user `049cb2fda8b74469945ac9aa7ae218cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:26:49.162134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:49.282114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1125/2213] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:26:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:26:49.291223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:49.292713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:49.324586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:26:49.350658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b96d4-0f9c-47b2-8258-b9cbcf9dda4b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:49.354193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:49.354786 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:49.355088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:49.355355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:49.357116 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:49.415160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:49.416494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:49.425202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1089/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:49.440231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:49.441895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:49.520717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a60c5-3f18-4496-8dbe-bbe8fb33eb3d None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:49.525628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:49.526264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:49.526547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:49.526837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] PATH_INFO: `/v3/projects/996248694ec54255a81168f3b06341cd` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:49.527996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=996248694ec54255a81168f3b06341cd)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:49.530484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:49.652823 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '996248694ec54255a81168f3b06341cd', 'request_id': 'req-e2fdd533-63c5-4474-a585-80000623b7a6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.659682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '996248694ec54255a81168f3b06341cd', 'request_id': 'req-e2fdd533-63c5-4474-a585-80000623b7a6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.665276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '996248694ec54255a81168f3b06341cd', 'request_id': 'req-e2fdd533-63c5-4474-a585-80000623b7a6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.764907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] The token cache is being invalidate because project 996248694ec54255a81168f3b06341cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:26:49.768450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.768450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:26:49.768450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1126/2215] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:26:49 2022] DELETE /identity/v3/projects/996248694ec54255a81168f3b06341cd => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:26:51.223055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:26:51.224493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:51.235671 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca6d56-bc56-47a4-81b0-24edb1d93bb1 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:26:51.284778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:26:51.285306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:26:51.285582 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:26:51.285852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:26:51.287527 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:26:51.350918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:26:51.352354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:26:51.362466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1090/2216] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:26:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:26:59.761028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:01.357917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=86 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:08.204480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:08.207968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:08.271681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdd533-63c5-4474-a585-80000623b7a6 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:08.276267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:08.276889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:08.277240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:08.277516 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] PATH_INFO: `/v3/users/e4d8414cfc954d2d81505e863b670096` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:08.278625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d8414cfc954d2d81505e863b670096)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:08.280695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:08.446545 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d8414cfc954d2d81505e863b670096'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.453407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d8414cfc954d2d81505e863b670096'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.454096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d8414cfc954d2d81505e863b670096'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.524125 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d8414cfc954d2d81505e863b670096'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.524745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d8414cfc954d2d81505e863b670096'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.542071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1127/2217] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:27:08 2022] DELETE /identity/v3/users/e4d8414cfc954d2d81505e863b670096 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:08.793960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:08.808281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c9d6b-6143-40e9-9342-2bc1af916757 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:08.864922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:08.866467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:08.866467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:08.866467 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] PATH_INFO: `/v3/projects/bc09774cd4634db792679d5276ea6a27` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:08.867383 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc09774cd4634db792679d5276ea6a27)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:08.869264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:08.968537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc09774cd4634db792679d5276ea6a27', 'request_id': 'req-5432af55-0731-4083-bb3e-063bafc818b6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.974541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc09774cd4634db792679d5276ea6a27', 'request_id': 'req-5432af55-0731-4083-bb3e-063bafc818b6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:08.980001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc09774cd4634db792679d5276ea6a27', 'request_id': 'req-5432af55-0731-4083-bb3e-063bafc818b6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.055597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] The token cache is being invalidate because project bc09774cd4634db792679d5276ea6a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:27:09.055890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.056513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5432af55-0731-4083-bb3e-063bafc818b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.057999 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1091/2218] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:27:08 2022] DELETE /identity/v3/projects/bc09774cd4634db792679d5276ea6a27 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:09.062010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:09.063414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:09.077312 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7dac35-65e8-4940-a729-0f251359ee04 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:09.133647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:09.134480 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:09.134692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:09.135002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] PATH_INFO: `/v3/users/049cb2fda8b74469945ac9aa7ae218cb` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:09.136258 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] RBAC: Authorizing `identity:delete_user(user_id=049cb2fda8b74469945ac9aa7ae218cb)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:09.138269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:09.162846 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:09.336958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '049cb2fda8b74469945ac9aa7ae218cb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.344951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '049cb2fda8b74469945ac9aa7ae218cb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.345639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '049cb2fda8b74469945ac9aa7ae218cb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.438710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '049cb2fda8b74469945ac9aa7ae218cb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.439368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '049cb2fda8b74469945ac9aa7ae218cb'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.458129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1128/2219] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:27:09 2022] DELETE /identity/v3/users/049cb2fda8b74469945ac9aa7ae218cb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:09.607430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:09.618504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:09.618504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:09.639666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:09.640868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:09.704013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1092/2220] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:27:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:09.730087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:09.730087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:09.796570 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dc8a9-81de-43bb-984a-1b3a1ea2455d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:09.800847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:09.801464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:09.801740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:09.802009 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] PATH_INFO: `/v3/projects/6453923e032b4d9185288d458faed4d7` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:09.803209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6453923e032b4d9185288d458faed4d7)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:09.805546 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:09.849019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:09.850444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:09.871474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:09.919179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371a986-5de8-400d-93ff-8bb8814e85cb None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:09.924321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:09.924981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:09.925268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:09.925579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:09.934892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:09.936539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e699a77-7f00-45c0-a927-548bb0464c30 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:09.943556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6453923e032b4d9185288d458faed4d7', 'request_id': 'req-8e399228-e9f3-463a-8547-b7797aa9d51c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.945192 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1093/2221] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:27:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:27:09.950422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6453923e032b4d9185288d458faed4d7', 'request_id': 'req-8e399228-e9f3-463a-8547-b7797aa9d51c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:09.956944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6453923e032b4d9185288d458faed4d7', 'request_id': 'req-8e399228-e9f3-463a-8547-b7797aa9d51c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:10.036417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] The token cache is being invalidate because project 6453923e032b4d9185288d458faed4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:27:10.036711 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:10.037385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-8e399228-e9f3-463a-8547-b7797aa9d51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:10.038957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1129/2222] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:27:09 2022] DELETE /identity/v3/projects/6453923e032b4d9185288d458faed4d7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:10.044675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.047921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.047921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.068420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:10.069705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:10.133988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1094/2223] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:10.141793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.142075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.142346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.167748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:10.168976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:10.177396 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1130/2224] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:10.181843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.193223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3636c3-4f5a-46c4-bea4-777debc4843b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.197945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.198550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.198852 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.199255 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.199903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:27:10.200205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.201794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.218806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1095/2225] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:27:10.219825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.231016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518875d5-4022-41ca-ab0f-97048d583d44 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.236200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.236956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.238157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.238157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.239010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.240991 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.326208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:10.334136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1131/2226] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:27:10 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:10.338162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.348978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25709d2-a72c-4812-b830-075b11b49e94 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.442104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.442936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.443307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.443655 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.444918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.447134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.539129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1096/2227] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:27:10 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:10.542847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.553904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b554e7-701b-4fb7-bbca-d27059b77b98 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.558918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.559522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.559798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.560070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.560847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.562551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.575691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1132/2228] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:27:10.579552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.590267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc95f52-dcb4-4503-afe5-e5a5fcf292d3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.595036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.595633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.595909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.596181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.596944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.598555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3095079-fd3f-4656-acb4-374ec1c80480 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.612756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1097/2229] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:27:10.616884 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.628477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4f805-877e-4fbb-9950-bd26ae1ff0d0 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.635331 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.636264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.636710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.637153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] PATH_INFO: `/v3/projects/0228237edca34990889f2074e63c33a5/users/debd9d6df31349d19b6897128102b345/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.654876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0228237edca34990889f2074e63c33a5, user_id=debd9d6df31349d19b6897128102b345, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.657918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:10.735403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1133/2230] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:27:10 2022] PUT /identity/v3/projects/0228237edca34990889f2074e63c33a5/users/debd9d6df31349d19b6897128102b345/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:27:10.749771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.750193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.750564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.783451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] MFA Rules not processed for user `debd9d6df31349d19b6897128102b345`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:10.785321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:10.790428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:10.874329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1098/2231] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:10.886855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:10.888700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:10.968620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1146b1-ec0c-47f7-aea0-3b310821e52c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:10.973323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:10.974049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:10.974500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:10.975293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:10.986992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:10.990100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c6b95-bd4c-4dee-bf2e-d222532a62b8 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:11.002276 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1134/2232] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:13.029512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:13.032409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc7da9-f9cc-4efe-902f-1eb87e4f7273 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:13.036433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:13.037018 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:13.037294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:13.037562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:13.039308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:13.098127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:13.099607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388726-1d84-422a-bfd0-cb557ab57f83 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:13.109959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1099/2233] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:20.997415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:23.104440 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=53 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:25.118388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c98d-0bcd-41c0-b996-bb06d539963f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:25.118917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c98d-0bcd-41c0-b996-bb06d539963f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:25.119021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452c98d-0bcd-41c0-b996-bb06d539963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:25.137748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c452c98d-0bcd-41c0-b996-bb06d539963f None None] MFA Rules not processed for user `1f737b4d0531425cbc803fcd15a3ea1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:25.138961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c452c98d-0bcd-41c0-b996-bb06d539963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:25.146085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1135/2234] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:28.783061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:28.783933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:28.784278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:28.785767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1100/2235] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:27:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:27:28.792143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a818-098d-4ebc-989c-25dccbffe5af None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:28.792670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a818-098d-4ebc-989c-25dccbffe5af None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:28.792767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4a818-098d-4ebc-989c-25dccbffe5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:28.826621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d0f4a818-098d-4ebc-989c-25dccbffe5af None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:28.828085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d0f4a818-098d-4ebc-989c-25dccbffe5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:28.923509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1136/2236] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:27:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:28.936884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:28.938570 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:29.007782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b083e18-1552-410b-aebf-0f7b7272e93c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:29.011334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:29.011996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.012264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.012533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.023036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:29.024462 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2d700-3cb1-4552-ac6f-ad32e29829bb service glance] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:29.037309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1101/2237] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:29.251000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.251287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.251564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] PATH_INFO: `/` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.252919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1137/2238] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:27:29.258091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc514c-7ed9-413c-8ccc-7cd87df010c2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.258374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc514c-7ed9-413c-8ccc-7cd87df010c2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.258710 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc514c-7ed9-413c-8ccc-7cd87df010c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.278627 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-cbbc514c-7ed9-413c-8ccc-7cd87df010c2 None None] MFA Rules not processed for user `822597c72e114b549c4b51c2aeaf63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:29.279935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-cbbc514c-7ed9-413c-8ccc-7cd87df010c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:29.334191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1102/2239] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:27:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:29.346791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:29.349030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:29.365299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25464a1-d412-4289-8699-241eb1331e35 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:29.368852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:29.370768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.370768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.370768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.386077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:29.386077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728477a4-0416-4aee-a2b8-89c37a179222 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:29.412358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1138/2240] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:29.490155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731633ec-582e-488d-ab64-485ff349c7a6 None None] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.490441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731633ec-582e-488d-ab64-485ff349c7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.490716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731633ec-582e-488d-ab64-485ff349c7a6 None None] PATH_INFO: `/` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.492106 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1103/2241] 10.222.0.42 () {54 vars in 921 bytes} [Wed Oct 26 11:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 26 11:27:29.497296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.497569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.497844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.518267 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] MFA Rules not processed for user `03aad2f885bc4dc183e934469676a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:29.519533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:29.529355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1139/2242] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Oct 26 11:27:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:29.540543 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731633ec-582e-488d-ab64-485ff349c7a6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:29.552626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731633ec-582e-488d-ab64-485ff349c7a6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:29.556480 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:29.557098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:29.557406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:29.557666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:29.569138 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:29.570585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea79174-3f0f-4305-85ed-819dece9abf2 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:29.582092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1104/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:31.356350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:31.426961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1af1-9001-45a4-9540-24c5fc66fff0 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:31.431701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:31.432338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:31.432624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:31.432899 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] PATH_INFO: `/v3/users/debd9d6df31349d19b6897128102b345` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:31.434040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debd9d6df31349d19b6897128102b345)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:31.436099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:31.614056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debd9d6df31349d19b6897128102b345'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:31.615412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:31.621838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debd9d6df31349d19b6897128102b345'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:31.622765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debd9d6df31349d19b6897128102b345'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:31.736024 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debd9d6df31349d19b6897128102b345'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:31.736765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debd9d6df31349d19b6897128102b345'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:31.761000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1140/2244] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:27:31 2022] DELETE /identity/v3/users/debd9d6df31349d19b6897128102b345 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:31.765430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:31.765900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:31.766165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:31.792011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:31.792011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:31.858159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1105/2245] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:31.867954 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:31.869352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:31.927423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdc381-1401-4bed-9968-b938841c6e35 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:31.931284 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:31.931894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:31.932243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:31.932615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:31.942380 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:31.943831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3494eb4-709b-4715-b786-fcfe7153b786 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:31.953400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1141/2246] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:27:32.365198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:32.366640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:32.379200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3df1f-6832-4bc0-8bd9-7ddd4f05d87c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:32.383843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:32.384647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.384993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.385197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] PATH_INFO: `/v3/projects/0228237edca34990889f2074e63c33a5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.386325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0228237edca34990889f2074e63c33a5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:32.388290 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:32.486684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:32.496531 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0228237edca34990889f2074e63c33a5', 'request_id': 'req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:32.503167 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0228237edca34990889f2074e63c33a5', 'request_id': 'req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:32.508716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0228237edca34990889f2074e63c33a5', 'request_id': 'req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:32.597628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] The token cache is being invalidate because project 0228237edca34990889f2074e63c33a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:27:32.598331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:32.598988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-173b176d-4b7c-4093-99e6-b3cb6aebb7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:27:32.600459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1106/2247] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:27:32 2022] DELETE /identity/v3/projects/0228237edca34990889f2074e63c33a5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:27:32.605365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.605853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.605950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.625914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:32.627142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:32.686913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1142/2248] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:32.694922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.695278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.700639 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.715759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:32.716973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:32.726339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1107/2249] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:32.735467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:32.745200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56518c-e28e-4825-bfb9-93c92ee01033 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:32.745715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:32.746323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.746600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.746916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.748059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:27:32.748059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:32.749443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:32.763029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1143/2250] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:27:32.769149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:32.779605 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b8bd-8178-4f10-972c-2a2b390cb507 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:32.784601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:32.785202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.786074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.786540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.787220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:32.789136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:32.902266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1108/2251] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:27:32 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:32.906406 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:32.917053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8b1cb-66e3-4d03-ac89-665c68c025f7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:32.965737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:32.972407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:32.973628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:32.974019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:32.974300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:32.974987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:32.976923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.066474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1144/2252] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:27:32 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:33.071067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:33.081210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82a78-367b-4f2f-9c7a-15494df46ef0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:33.086673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:33.092677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.092677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.092677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.092677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:33.092677 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.099983 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1109/2253] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:27:33.104168 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:33.116160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2cb76-969a-403c-8404-765da16b2345 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:33.125936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:33.125936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.125936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.125936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.126303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:33.126303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a069073-ed25-4252-8e99-eea0040fe3bd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.136398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1145/2254] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:27:33.140636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:33.151622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb92e77-cd15-42c2-9eeb-0bbc45aef6fc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:33.156310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:33.156900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.157186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.157613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] PATH_INFO: `/v3/projects/a975701f3c69481b874e106eb5d4edcd/users/e7088cf9374c4e7d8e60c34a782fb42f/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.173087 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.173369 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.173645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.176137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a975701f3c69481b874e106eb5d4edcd, user_id=e7088cf9374c4e7d8e60c34a782fb42f, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:33.178850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806afa81-7f8b-4c49-bb50-199bf8fcd00e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.193453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:33.195249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:33.248728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1146/2255] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:27:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:33.261161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:33.262471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:33.267150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1110/2256] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:27:33 2022] PUT /identity/v3/projects/a975701f3c69481b874e106eb5d4edcd/users/e7088cf9374c4e7d8e60c34a782fb42f/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:33.273061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e293-923d-454b-b074-616527a67722 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.273345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e293-923d-454b-b074-616527a67722 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.273716 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795e293-923d-454b-b074-616527a67722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.293420 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1795e293-923d-454b-b074-616527a67722 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:33.294648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1795e293-923d-454b-b074-616527a67722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:33.343042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887caa6-6299-4dcf-8216-09f988665047 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:33.361778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1111/2257] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:33.375308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795e293-923d-454b-b074-616527a67722 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:33.376917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1795e293-923d-454b-b074-616527a67722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:33.414069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:33.414749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.415882 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.416166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.439173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795e293-923d-454b-b074-616527a67722 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:33.441784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:33.443873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:33.444622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:33.444971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:33.445343 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:33.465779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:33.467336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505a28b-0d79-4dd5-a1e3-eadf4bf5d875 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.477691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1112/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:33.482646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:33.484373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:33.493389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1147/2259] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:27:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:27:40.458873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:40.459428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:40.459534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:40.469210 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:40.503217 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] MFA Rules not processed for user `e7088cf9374c4e7d8e60c34a782fb42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:40.505368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:40.589625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1113/2260] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:40.590538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:40.603014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87736a41-f066-4063-98e6-33d7ae30aaa8 service swift] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:40.607691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:40.608310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:40.608588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:40.608851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:40.609489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:40.611457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:40.826524 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1148/2261] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:27:40 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:40.828469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:40.840086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83c874-3c65-4eab-a9d3-477a48d89850 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:40.914877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:40.915620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:40.915990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:40.916322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:40.917041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:40.919130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:41.038834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1114/2262] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:27:40 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:27:41.043481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:41.055223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a3cf2-8485-42bd-a727-ac1210ff2b03 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:41.060513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:41.061474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:41.061876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:41.062415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:41.063219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:41.065131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:41.078806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1149/2263] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:27:41.082941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:41.094551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375cc6-6ba4-4e98-829c-b6b5780cc083 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:41.100177 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:41.103076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:41.103076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:41.103076 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] PATH_INFO: `/v3/projects/85c679ebc0954f48a6fd364ee5924df5/users/988ee615f0104dc4b6e6a4d617093093/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:41.125664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c679ebc0954f48a6fd364ee5924df5, user_id=988ee615f0104dc4b6e6a4d617093093, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:41.132730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:41.211667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1115/2264] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:27:41 2022] PUT /identity/v3/projects/85c679ebc0954f48a6fd364ee5924df5/users/988ee615f0104dc4b6e6a4d617093093/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:41.215728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:41.227075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b5349-0d6a-4cc5-b12c-472e1aa49649 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:41.296383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:41.296988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:41.297263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:41.297532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:41.298163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:41.299768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716b9d1-d049-4244-a779-91f786962019 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:41.310860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1150/2265] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:27:41.315472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:41.345837 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b3a82c-d250-44b1-9689-c8387988a30b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:41.350595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:41.360011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:41.360011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:41.360011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] PATH_INFO: `/v3/projects/85c679ebc0954f48a6fd364ee5924df5/users/988ee615f0104dc4b6e6a4d617093093/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:41.360011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c679ebc0954f48a6fd364ee5924df5, user_id=988ee615f0104dc4b6e6a4d617093093, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:41.360011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:41.370868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1116/2266] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:27:41 2022] PUT /identity/v3/projects/85c679ebc0954f48a6fd364ee5924df5/users/988ee615f0104dc4b6e6a4d617093093/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:27:47.833110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881efe3-d398-410e-ac37-e7a7ed53835b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:47.834072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881efe3-d398-410e-ac37-e7a7ed53835b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:47.834623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881efe3-d398-410e-ac37-e7a7ed53835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:47.890317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5881efe3-d398-410e-ac37-e7a7ed53835b None None] MFA Rules not processed for user `988ee615f0104dc4b6e6a4d617093093`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:47.892040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5881efe3-d398-410e-ac37-e7a7ed53835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:47.978096 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1151/2267] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:27:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:47.986880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:47.990244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:48.059305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:48.070935 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270814bb-175d-4425-a274-8f4a2bc28d69 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:48.077347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:48.077347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:48.077347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:48.077347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:48.134498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:48.141022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:48.146314 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1117/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:48.428238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:48.428556 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:48.428834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:48.448086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:48.449319 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:48.454553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:27:48.504122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1152/2269] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:27:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:27:48.516474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:48.581274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d204b7eb-01ee-4488-8762-65a2d9239c92 service glance] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:48.586834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:48.587456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:48.587870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:48.588152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:48.596709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:48.598330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3d063-ab1f-420d-a565-632be6ec5e50 service swift] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:48.607248 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1118/2270] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:27:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:27:55.931336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:55.995298 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f6248-67ca-4b6c-bb8c-598cad360d75 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:55.999076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:55.999761 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:56.000119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:56.000452 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:56.002263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:56.074946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:56.076647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:56.088238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1153/2271] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:27:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:27:57.300293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040798ee-950e-499f-ab9a-7f744ca6e774 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:57.301267 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040798ee-950e-499f-ab9a-7f744ca6e774 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:57.302349 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040798ee-950e-499f-ab9a-7f744ca6e774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:57.327095 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-040798ee-950e-499f-ab9a-7f744ca6e774 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:27:57.328968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-040798ee-950e-499f-ab9a-7f744ca6e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:27:57.337555 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1119/2272] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:27:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:27:57.354902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:27:57.373988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0201ac-6fd2-4619-9427-72dad76ec370 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:27:57.381053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:27:57.381871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:27:57.382328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:27:57.386213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:27:57.401969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:27:57.405383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:27:57.419200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1154/2273] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:27:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 26 11:28:00.065604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedf99a-a19a-4d17-8fd2-54536ddaf0bb None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:00.066093 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedf99a-a19a-4d17-8fd2-54536ddaf0bb None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:00.066193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fedf99a-a19a-4d17-8fd2-54536ddaf0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:00.086034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4fedf99a-a19a-4d17-8fd2-54536ddaf0bb None None] MFA Rules not processed for user `e7088cf9374c4e7d8e60c34a782fb42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:00.087245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4fedf99a-a19a-4d17-8fd2-54536ddaf0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:00.093966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1120/2274] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:28:00.491978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:00.493415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:00.505740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129b45-80ab-42c7-a263-305654797e9d service swift] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:00.509283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:00.509812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:00.510093 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:00.510362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:00.512265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:00.565818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:00.589935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:00.591590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:00.603412 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1155/2275] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:03.640439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:03.640954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:03.641071 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:03.661126 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:03.662345 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:03.669597 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1121/2276] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Oct 26 11:28:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:03.683596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:03.694563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ebd1a-c350-40cd-aaea-995ebd7e28f2 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:03.699202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:03.699819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:03.700094 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:03.700363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:03.708766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:03.710402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:03.718231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1156/2277] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:28:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:28:04.086158 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:04.096705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f5b94-b071-4524-93f7-061eb0efa1ce None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:04.101007 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:04.101737 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:04.102073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:04.102407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:04.105123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:04.120609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:04.122070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:04.135560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1122/2278] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:13.715055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=16 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:14.129069 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=75 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:42.571188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:42.580414 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b944560-af49-478d-9482-9f01f406f8b1 service swift] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:42.584080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:42.584665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:42.584940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:42.585208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:42.586898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:42.651039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:42.652460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:42.661999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1157/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:44.331335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:44.341272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4045-03ad-487f-872a-aadf979182df service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:44.345159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:44.345775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:44.346058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:44.346336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:44.362365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:44.362365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:44.369127 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1123/2280] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:28:46.224220 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:46.226678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:46.239264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762089e4-37ce-40b5-80ba-4181ff8c9af2 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:46.244462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:46.245289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:46.245601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:46.245879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] PATH_INFO: `/v3/users/e031dd9da25146eebb8813dc8bad8b85` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:46.260588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e031dd9da25146eebb8813dc8bad8b85)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:46.262494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:46.432418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e031dd9da25146eebb8813dc8bad8b85'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:46.439705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e031dd9da25146eebb8813dc8bad8b85'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:46.440379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e031dd9da25146eebb8813dc8bad8b85'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:46.519937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e031dd9da25146eebb8813dc8bad8b85'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:46.520580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e031dd9da25146eebb8813dc8bad8b85'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:46.539590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1158/2281] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:28:46 2022] DELETE /identity/v3/users/e031dd9da25146eebb8813dc8bad8b85 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:46.751315 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:46.760223 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be20f5a-e890-4280-9a59-9f4650a96ecf service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:46.793739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:46.795639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:46.825331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:46.826774 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:46.828132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:46.828872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:46.830220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:46.846254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:46.848011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d6223-e310-4c01-8fea-54d0197bf5fd service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:46.862356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1124/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:28:46.882845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:46.887786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0d461-8cad-48b7-805f-a37165727dd7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:46.892868 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:46.894191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:46.894805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:46.895118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] PATH_INFO: `/v3/projects/724121c2144843b7adc7e17312613c9d` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:46.906930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=724121c2144843b7adc7e17312613c9d)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:46.908956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.020184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724121c2144843b7adc7e17312613c9d', 'request_id': 'req-5122abef-fa86-48db-879b-366005d1df06'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:47.028301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724121c2144843b7adc7e17312613c9d', 'request_id': 'req-5122abef-fa86-48db-879b-366005d1df06'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:47.036400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724121c2144843b7adc7e17312613c9d', 'request_id': 'req-5122abef-fa86-48db-879b-366005d1df06'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:47.132043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] The token cache is being invalidate because project 724121c2144843b7adc7e17312613c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:28:47.132342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:47.133030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-5122abef-fa86-48db-879b-366005d1df06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:47.134877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1159/2283] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:28:46 2022] DELETE /identity/v3/projects/724121c2144843b7adc7e17312613c9d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:47.139732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.140268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.140546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.170227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:47.171079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:47.246028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1125/2284] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:47.253876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.254921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.255203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.276901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:47.279030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:47.291471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1160/2285] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:47.295794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.306024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe0976-26d7-4604-9428-9a1eaa121c92 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.316229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.316995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.317276 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.317558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.318211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:28:47.318628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.320798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.345224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1126/2286] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:28:47.349584 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.364450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8adbd-5a92-4214-b0d5-43d9714c8957 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.371491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.372108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.372479 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.372648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.373286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.375241 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.541700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1161/2287] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:28:47 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:47.543030 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.555381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d339-f0c8-42cd-a919-3a5db1b7ad9a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.619246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.621051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.621051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.621051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.621051 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.623112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.710288 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1127/2288] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:28:47 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:47.711645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.722914 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb720e5-5b13-437e-9150-da8d53c7f088 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.727655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.730038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.730038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.730038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.730346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.731688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.741714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1162/2289] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:28:47.745835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.756299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b20dd-befd-480a-9ee2-288477935247 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.760807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.761422 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.761698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.761967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.762599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.764213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48ff46-8a91-4e79-a3ff-664299b0d1d1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.765482 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:47.777813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1128/2290] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:28:47.781957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:47.793038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ec90f-78eb-4d90-bf37-bb25b3230750 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:47.799201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:47.799834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.800139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.800280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] PATH_INFO: `/v3/projects/127e058a793441948288b4b28144541f/users/c2d487cb0b1444f7afb4004486ec7c42/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.816604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] RBAC: Authorizing `identity:create_grant(project_id=127e058a793441948288b4b28144541f, user_id=c2d487cb0b1444f7afb4004486ec7c42, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:47.819311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:47.904444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1163/2291] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:28:47 2022] PUT /identity/v3/projects/127e058a793441948288b4b28144541f/users/c2d487cb0b1444f7afb4004486ec7c42/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:47.916728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:47.917013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:47.917287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:47.945635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] MFA Rules not processed for user `c2d487cb0b1444f7afb4004486ec7c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:47.946881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:48.012082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1129/2292] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:48.023494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:48.025202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:48.101280 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:48.107373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e7b73-c945-4eb4-bee9-1420dbee826e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:48.112929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:48.117227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:48.117227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:48.117227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:48.121601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:48.123190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5f9dc-aa5c-4f35-85cd-15294726fa26 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:48.131675 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1164/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:48.347675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:48.410666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5629b6-7352-4e04-9d49-1b288a65cc7a None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:48.415253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:48.416161 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:48.416275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:48.416470 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] PATH_INFO: `/v3/users/c2d487cb0b1444f7afb4004486ec7c42` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:48.417585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d487cb0b1444f7afb4004486ec7c42)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:48.419530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:48.617730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d487cb0b1444f7afb4004486ec7c42'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:48.627212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d487cb0b1444f7afb4004486ec7c42'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:48.627919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d487cb0b1444f7afb4004486ec7c42'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:48.701219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d487cb0b1444f7afb4004486ec7c42'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:48.701864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d487cb0b1444f7afb4004486ec7c42'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:48.720063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1130/2294] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:28:48 2022] DELETE /identity/v3/users/c2d487cb0b1444f7afb4004486ec7c42 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:48.724788 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:48.725273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:48.725399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:48.745694 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:48.747048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:48.808472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1165/2295] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:48.818236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:48.819674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:48.828784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:48.878433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0ae58-26ec-4160-bc96-9fd83262f7de admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:48.881995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:48.882604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:48.882911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:48.883188 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:48.892564 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:48.893977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9023aa8-7a07-4e1a-ab4f-902ba54c34e5 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:48.903472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1131/2296] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:49.434030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:49.435514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:49.449302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adbdd2-418f-4699-be2f-114da52f06d9 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:49.453795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:49.454504 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:49.454793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:49.455538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] PATH_INFO: `/v3/projects/127e058a793441948288b4b28144541f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:49.456497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] RBAC: Authorizing `identity:delete_project(project_id=127e058a793441948288b4b28144541f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:49.458400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:49.583243 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127e058a793441948288b4b28144541f', 'request_id': 'req-13fde78e-103f-493b-8f73-085ff183579c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:49.590279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127e058a793441948288b4b28144541f', 'request_id': 'req-13fde78e-103f-493b-8f73-085ff183579c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:49.595889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127e058a793441948288b4b28144541f', 'request_id': 'req-13fde78e-103f-493b-8f73-085ff183579c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:49.660821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] The token cache is being invalidate because project 127e058a793441948288b4b28144541f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:28:49.666759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:49.666759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-13fde78e-103f-493b-8f73-085ff183579c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:28:49.666759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1166/2297] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:28:49 2022] DELETE /identity/v3/projects/127e058a793441948288b4b28144541f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:49.668829 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:49.669109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:49.669409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:49.690598 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:49.691855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:49.758836 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1132/2298] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:49.767182 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:49.767649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:49.767760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:49.788465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:49.789716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:49.797618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1167/2299] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:28:49.801745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:49.812800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ee47c-e93a-44e4-baa1-f41f2fc9fa9d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:49.817980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:49.818581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:49.819139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:49.819379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:49.820035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:28:49.820365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:49.821923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:49.835694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1133/2300] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:28:49.840202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:49.851542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcd6f0-9a1b-4f15-8cc4-e6f0527f005a None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:49.864946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:49.865571 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:49.865898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:49.866173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:49.866805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:49.868801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:49.966712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1168/2301] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:28:49 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:49.970788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:49.981914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f39bec-c207-49cf-9148-a9f52fbd3b41 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.049642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.050667 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.051102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.051502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.052289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.054361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.230887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1134/2302] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:28:49 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:50.236399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.251040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97010f32-908a-4b8e-9f61-19665ccbe28e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.257082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.257735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.258043 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.258654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.259385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.261588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.272310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1169/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:28:50.281596 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.292154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a21d-8b74-4808-a9d5-02997dcda40d admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.297038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.297651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.297927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.298195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.298858 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.300498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1205c1f0-04ff-407c-af7c-cd0161c7c047 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.312502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1135/2304] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:28:50.316460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.328732 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50651-b5bc-47c0-914c-079d8380121b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.333633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.336254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.336254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.336254 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] PATH_INFO: `/v3/projects/ef2ede3781994ac59f6eee87640434d1/users/1612fdcc19464443baf578778557d0a3/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.351487 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2ede3781994ac59f6eee87640434d1, user_id=1612fdcc19464443baf578778557d0a3, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.355049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.433536 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1170/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:28:50 2022] PUT /identity/v3/projects/ef2ede3781994ac59f6eee87640434d1/users/1612fdcc19464443baf578778557d0a3/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:28:50.444723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.447997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.447997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.472084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:50.475487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] MFA Rules not processed for user `1612fdcc19464443baf578778557d0a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:50.476788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:50.548887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1136/2306] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:50.553251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.563874 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:50.614787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc717fe-c4c7-45c5-b0fe-837c0f2a70b1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.619990 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.620572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.620847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.621116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.621772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.623716 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.703871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1171/2307] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:28:50 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:50.707903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.718547 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637195e-ed66-4548-a249-4a5e3d7eff1e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.780067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.780747 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.781027 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.781306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.781944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.783893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.909230 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1137/2308] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:28:50 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:28:50.913064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.923128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3796405-32a5-4552-a91a-b4fda080eb85 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.927590 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.928174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.928447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.928712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.929337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.930953 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:50.941771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1172/2309] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:28:50.945796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:50.956767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9798fd97-d4f8-4def-ba55-23669fc2039b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:50.961595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:50.962251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:50.962571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:50.962979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] PATH_INFO: `/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e/users/3250e069fded4522947239c6fc3c7d83/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:50.977704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=856ce3ffbdcc43ad9ac5891781aac01e, user_id=3250e069fded4522947239c6fc3c7d83, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:50.980889 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:51.085691 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1138/2310] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:28:50 2022] PUT /identity/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e/users/3250e069fded4522947239c6fc3c7d83/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:51.089756 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:51.099625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3f28b-e87e-4e6d-aca8-c61c73e04e90 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:51.155491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:51.156083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:51.156361 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:51.156629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:51.157256 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:51.158912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd343e06-a51f-4ebf-8454-20364b80b49d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:51.168808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1173/2311] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:28:51.172953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:51.183960 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36b3ed-ddc8-4450-80ed-7bd35a7f5fce admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:51.188576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:51.189218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:51.189501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:51.189777 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] PATH_INFO: `/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e/users/3250e069fded4522947239c6fc3c7d83/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:51.191729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=856ce3ffbdcc43ad9ac5891781aac01e, user_id=3250e069fded4522947239c6fc3c7d83, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:51.194404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:51.205435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1139/2312] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:28:51 2022] PUT /identity/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e/users/3250e069fded4522947239c6fc3c7d83/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:28:51.216785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:51.217098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:51.217371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:51.245569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] MFA Rules not processed for user `3250e069fded4522947239c6fc3c7d83`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:28:51.246784 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:51.313366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1174/2313] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:28:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:28:51.325798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:51.327548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:28:51.382669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7c948-3dba-471a-bddf-2aa2fcf204ab admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:51.386878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:51.387471 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:51.387743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:51.388013 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:51.396833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:51.398492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:51.402350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:51.408234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1140/2314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:28:51.621299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:28:51.627250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:28:51.632332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530682-6087-472e-af9e-8db4b182bbff None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:28:51.636688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:28:51.637304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:28:51.637582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:28:51.637855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:28:51.698474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:28:51.699943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:28:51.708612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1175/2315] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:28:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:01.702961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:01.917995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=26 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:05.757620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:05.766763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246bb9e-a62c-4cdc-a923-c417e52f3181 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:05.770560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:05.771397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:05.771681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:05.771954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:05.774016 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:05.856745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:05.858678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:05.877502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1141/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:09.253456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:09.255878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:09.342704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e98f4a7-a599-4ce4-9a88-3b97d96fa4c1 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:09.350120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:09.350968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:09.351465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:09.351901 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:09.354511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:09.369712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:09.371424 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:09.383836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1176/2317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:15.874719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=41 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:16.069144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:16.070589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:16.126159 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affdcb-b3e5-483e-bd82-bc0ab7a27c95 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:16.129719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:16.130312 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:16.130832 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:16.131129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:16.132877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:16.194800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:16.196250 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:16.207222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1142/2318] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:19.378979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=34 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:19.899382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:19.900888 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:19.916089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4afdcb-3fbe-480e-b6f9-8a3f1a1eded6 service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:19.920925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:19.921733 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:19.922040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:19.922427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] PATH_INFO: `/v3/users/41276c93e476436791b89ab8a0d4537d` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:19.923583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41276c93e476436791b89ab8a0d4537d)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:19.925498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:20.105006 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41276c93e476436791b89ab8a0d4537d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.112230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41276c93e476436791b89ab8a0d4537d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.112885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41276c93e476436791b89ab8a0d4537d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.175375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41276c93e476436791b89ab8a0d4537d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.175988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ec00ad1c-194b-412c-86fd-4999b57dff19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41276c93e476436791b89ab8a0d4537d'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.193769 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1177/2319] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:29:19 2022] DELETE /identity/v3/users/41276c93e476436791b89ab8a0d4537d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:20.459614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:20.470683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf559c1-4e8a-42ce-b517-92179f37d931 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:20.528274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:20.528917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:20.529297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:20.529584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] PATH_INFO: `/v3/projects/ab29bf15969d46da9670efa2b52e6676` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:20.530678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab29bf15969d46da9670efa2b52e6676)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:20.533130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:20.621297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab29bf15969d46da9670efa2b52e6676', 'request_id': 'req-3020e416-121d-4f1a-bcd0-f01c13b68735'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.623993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:20.628201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab29bf15969d46da9670efa2b52e6676', 'request_id': 'req-3020e416-121d-4f1a-bcd0-f01c13b68735'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.633694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab29bf15969d46da9670efa2b52e6676', 'request_id': 'req-3020e416-121d-4f1a-bcd0-f01c13b68735'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.712139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] The token cache is being invalidate because project ab29bf15969d46da9670efa2b52e6676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:29:20.712364 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.712989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3020e416-121d-4f1a-bcd0-f01c13b68735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:20.714550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1143/2320] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:29:20 2022] DELETE /identity/v3/projects/ab29bf15969d46da9670efa2b52e6676 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:20.720040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:20.720542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:20.720651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:20.742293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:20.743547 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:20.803083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1178/2321] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:29:20.811257 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:20.811546 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:20.811821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:20.832193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:20.833388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:20.841292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1144/2322] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:20.851864 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:20.855812 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c3458-7cee-46b1-b15b-955fcc6ac1d4 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:20.860304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:20.860887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:20.861161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:20.861432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:20.862063 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:29:20.862362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:20.863974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:20.875580 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1179/2323] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:29:20.879878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:20.889237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce27b-8b66-4e7c-a46e-334db203212b None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:20.894448 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:20.895094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:20.895375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:20.895648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:20.896277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:20.898432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.014177 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1145/2324] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:29:20 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:29:21.018988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:21.029609 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0057dd-1819-4d01-8a1b-eae63bc895f8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:21.084759 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:21.085434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.085715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.085989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.086624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:21.088573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.097070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:21.206681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1180/2325] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:29:21 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:21.211017 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:21.221204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ee0b-7c23-4c79-87b9-b68e6a2c52d2 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:21.225758 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:21.233309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.233309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.233309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.233309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:21.233309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.239060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1146/2326] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:21.243136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:21.253533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3e109-76c6-4963-962f-0051bad07e81 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:21.258154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:21.258735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.259048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.259320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.259982 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:21.261555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42292ab8-23b1-4ed1-86da-4e82bbb85bd8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.272762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1181/2327] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:21.276991 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:21.286809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e91f15-0b36-4c69-bab5-f0c468391098 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:21.291332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:21.291971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.292249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.292518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] PATH_INFO: `/v3/projects/b78491741fc7450b932b7a43e5d3bb92/users/80087d6fbe284b02b20c64ce3237d3b5/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.307587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78491741fc7450b932b7a43e5d3bb92, user_id=80087d6fbe284b02b20c64ce3237d3b5, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:21.310211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.369751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1147/2328] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:21 2022] PUT /identity/v3/projects/b78491741fc7450b932b7a43e5d3bb92/users/80087d6fbe284b02b20c64ce3237d3b5/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:21.374796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac3d1-b601-433a-9ddb-80bc3d31226d None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.376778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac3d1-b601-433a-9ddb-80bc3d31226d None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.376778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eac3d1-b601-433a-9ddb-80bc3d31226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.395156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-87eac3d1-b601-433a-9ddb-80bc3d31226d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:21.396345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-87eac3d1-b601-433a-9ddb-80bc3d31226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:21.452634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1182/2329] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:21.462756 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:21.464465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:21.520719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02875-903e-4a92-9169-cbb4c14a5a2e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:21.524664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:21.525197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:21.525476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:21.525749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:21.534548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:21.535993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:21.545115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1148/2330] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:26.663767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037619-e191-4533-b674-08782a6f801c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:26.664248 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037619-e191-4533-b674-08782a6f801c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:26.664349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a037619-e191-4533-b674-08782a6f801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:26.699180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-5a037619-e191-4533-b674-08782a6f801c None None] MFA Rules not processed for user `80087d6fbe284b02b20c64ce3237d3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:26.700332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5a037619-e191-4533-b674-08782a6f801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:26.776073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1183/2331] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:26.793643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:26.795647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:26.872808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76640aa-9527-45e1-bd6a-18713aeb647c service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:26.879236 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:26.879996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:26.880344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:26.880775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:26.889750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:26.891944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:26.902465 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1149/2332] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:29.534177 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:29.534701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:29.534805 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:29.556287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] MFA Rules not processed for user `80087d6fbe284b02b20c64ce3237d3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:29.560511 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:29.575898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1184/2333] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:29.930947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:29.933108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:29.944358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:29.950586 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cced90f-d8cd-483d-914d-552e0d7b5f82 service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:29.956457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:29.957143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:29.957573 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:29.958426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:29.960317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:29.973041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:29.974748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:29.983908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1150/2334] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:30.454637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:30.456162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:30.539602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:30.543653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28ee0-2eb2-40e2-a8dd-143610683733 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:30.549122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:30.549747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:30.550034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:30.550306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:30.552154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:30.564313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:30.565916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:30.576679 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1185/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:29:40.730145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:40.758855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:44.148144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:44.151460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:44.167715 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c9ef-fea7-4106-997b-3aef597bab9f service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:44.173438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:44.174036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:44.174318 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:44.174593 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:44.176431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:44.188713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:44.190313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d4a030-b652-461b-9625-8d6a4c963bfb service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:44.200949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1151/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:47.426641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:47.428086 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:47.441665 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb3083-7b83-41af-95a7-4dc8810f5750 service placement] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:47.446335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:47.447019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:47.447304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:47.447618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] PATH_INFO: `/v3/users/1612fdcc19464443baf578778557d0a3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:47.448730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1612fdcc19464443baf578778557d0a3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:47.450672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:47.655937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1612fdcc19464443baf578778557d0a3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:47.665136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1612fdcc19464443baf578778557d0a3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:47.665939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1612fdcc19464443baf578778557d0a3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:47.753392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1612fdcc19464443baf578778557d0a3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:47.754105 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1612fdcc19464443baf578778557d0a3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:47.773098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1186/2337] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:29:47 2022] DELETE /identity/v3/users/1612fdcc19464443baf578778557d0a3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:47.777914 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:47.778398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:47.778495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:47.797525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:47.799291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:47.861044 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1152/2338] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:47.869755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:47.871208 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:47.931069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da26f0e-221e-4e0d-b37a-67fec0906cbc admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:47.934764 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:47.935434 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:47.935715 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:47.935987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:47.946119 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:47.947594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:47.960143 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1187/2339] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:47.974033 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:47.975571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:48.057953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94291fe-d6de-4891-9506-8678a1ab2bef None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:48.064184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:48.064868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:48.065150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:48.065432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:48.067254 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:48.079796 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:48.081361 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:48.092764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1153/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:29:48.442358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:48.444460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:48.460212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fb1bb9-a6e9-4a9f-8d43-d2a23175fe3a service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:48.466192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:48.466610 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:48.466763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:48.468032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] PATH_INFO: `/v3/projects/ef2ede3781994ac59f6eee87640434d1` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:48.469904 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2ede3781994ac59f6eee87640434d1)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:48.472106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:48.621754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2ede3781994ac59f6eee87640434d1', 'request_id': 'req-a26cd425-db6b-459b-9d92-3e997015018a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:48.630838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2ede3781994ac59f6eee87640434d1', 'request_id': 'req-a26cd425-db6b-459b-9d92-3e997015018a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:48.638058 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2ede3781994ac59f6eee87640434d1', 'request_id': 'req-a26cd425-db6b-459b-9d92-3e997015018a'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:48.708849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:48.713773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] The token cache is being invalidate because project ef2ede3781994ac59f6eee87640434d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:29:48.714142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:48.714757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:48.718026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1188/2341] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:29:48 2022] DELETE /identity/v3/projects/ef2ede3781994ac59f6eee87640434d1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:29:48.721270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:48.735102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8119a-e042-42d8-9166-2423dd1dbb84 service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:48.796015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:48.796741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:48.796912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:48.797186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] PATH_INFO: `/v3/users/3250e069fded4522947239c6fc3c7d83` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:48.799130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3250e069fded4522947239c6fc3c7d83)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:48.801008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:48.816643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:49.026595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3250e069fded4522947239c6fc3c7d83'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.039530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3250e069fded4522947239c6fc3c7d83'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.039530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3250e069fded4522947239c6fc3c7d83'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.121417 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3250e069fded4522947239c6fc3c7d83'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.122121 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c3c16cd4-ea26-405d-8b39-4e119bfbee3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3250e069fded4522947239c6fc3c7d83'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.141293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1154/2342] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:29:48 2022] DELETE /identity/v3/users/3250e069fded4522947239c6fc3c7d83 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:49.545047 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:49.546532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:49.559857 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26cd425-db6b-459b-9d92-3e997015018a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:49.621576 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:49.622195 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:49.622471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:49.622738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] PATH_INFO: `/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:49.623853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] RBAC: Authorizing `identity:delete_project(project_id=856ce3ffbdcc43ad9ac5891781aac01e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:49.625728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:49.726735 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '856ce3ffbdcc43ad9ac5891781aac01e', 'request_id': 'req-c8b7fb9d-cca5-481f-bd21-68d87b8282af'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.733034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '856ce3ffbdcc43ad9ac5891781aac01e', 'request_id': 'req-c8b7fb9d-cca5-481f-bd21-68d87b8282af'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.739373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '856ce3ffbdcc43ad9ac5891781aac01e', 'request_id': 'req-c8b7fb9d-cca5-481f-bd21-68d87b8282af'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.823563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] The token cache is being invalidate because project 856ce3ffbdcc43ad9ac5891781aac01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:29:49.823796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.824441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c8b7fb9d-cca5-481f-bd21-68d87b8282af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:49.825934 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1189/2343] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:29:49 2022] DELETE /identity/v3/projects/856ce3ffbdcc43ad9ac5891781aac01e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:49.831338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:49.833741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:49.833741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:49.851005 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:49.852189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:49.909337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1155/2344] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:49.917206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:49.917488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:49.917760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:49.937603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:49.939342 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:49.947200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1190/2345] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:49.951218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:49.961175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe99cbe-f798-4654-8791-9c0c2e31191e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:49.965766 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:49.966346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:49.966626 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:49.966931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:49.967599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:29:49.967900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:49.969489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:49.980683 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1156/2346] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:29:49.984754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:49.993847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d38bce-3672-484c-b0d0-6fe4281feafe None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:49.998453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:49.999132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:49.999411 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:49.999683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.000302 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.002663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.080517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1191/2347] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:29:49 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:50.084709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.095047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd67791-fddc-40a0-9609-427909052bcb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.151780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.152289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.152569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.152840 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.153450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.155384 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.238932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1157/2348] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:29:50 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:50.243074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.254022 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f08da-78b1-4806-acdf-911a8b47ac2e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.258745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.259378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.259656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.259927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.260651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.262264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.272283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1192/2349] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:50.276237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.286058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b55fe-0c3e-48e8-8f1c-861f949b6192 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.290560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.291169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.291514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.291784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.292408 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.294082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936bf27-f0eb-4baf-a207-74c8e954453f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.296103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:50.305197 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1158/2350] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:50.309833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.320350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbde61-4fc4-4ef9-b121-ba24d40895f3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.324999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.325629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.325909 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.326184 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] PATH_INFO: `/v3/projects/c4ad555ef7214224ab0a3eba086cda22/users/bdd9a716dd5a4da48a6cabf56e30e0cc/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.340457 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ad555ef7214224ab0a3eba086cda22, user_id=bdd9a716dd5a4da48a6cabf56e30e0cc, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.343052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.435595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1193/2351] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:50 2022] PUT /identity/v3/projects/c4ad555ef7214224ab0a3eba086cda22/users/bdd9a716dd5a4da48a6cabf56e30e0cc/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:29:50.446898 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.447181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.447481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.476086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] MFA Rules not processed for user `bdd9a716dd5a4da48a6cabf56e30e0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:50.477282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:50.538720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1159/2352] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:50.543173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.569743 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:50.603441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512037d-ab9c-4cf1-9895-70780c251df2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.607923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.608514 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.608790 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.609055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.609678 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.612273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.681806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1194/2353] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:29:50 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:29:50.685868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.696432 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3efe1-57a0-4253-9d31-e3e04eaa9b34 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.752266 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.752926 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.753202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.753469 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.754102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.756034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.844740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1160/2354] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:29:50 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:50.849219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.859530 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b342c-221d-4949-829b-f2f0d9f5e211 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.864107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.864696 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.864970 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.865236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.865863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.867513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:50.877718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1195/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:50.883672 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:50.894107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f639077-eecc-4249-b88f-4653212b52fd admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:50.899148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:50.899902 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:50.900213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:50.900485 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] PATH_INFO: `/v3/projects/724495bba7eb4c40962977360d1e47d6/users/34b715a5234240f688a717b4ef7236c4/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:50.915534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724495bba7eb4c40962977360d1e47d6, user_id=34b715a5234240f688a717b4ef7236c4, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:50.918190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:51.022381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1161/2356] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:50 2022] PUT /identity/v3/projects/724495bba7eb4c40962977360d1e47d6/users/34b715a5234240f688a717b4ef7236c4/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:51.026643 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:51.037076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbc101-0db2-428b-ab54-6c72b6b16348 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:51.092420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:51.093016 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:51.093317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:51.093588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:51.094216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:51.096521 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe6800-89a7-4412-a62b-f497bd0f226c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:51.106014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1196/2357] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:29:51.115704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:51.120489 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f786e6-84a9-4fb5-af07-2b197938df27 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:51.125291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:51.126479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:51.126759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:51.127053 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] PATH_INFO: `/v3/projects/724495bba7eb4c40962977360d1e47d6/users/34b715a5234240f688a717b4ef7236c4/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:51.129037 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724495bba7eb4c40962977360d1e47d6, user_id=34b715a5234240f688a717b4ef7236c4, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:51.131662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:51.142302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1162/2358] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:51 2022] PUT /identity/v3/projects/724495bba7eb4c40962977360d1e47d6/users/34b715a5234240f688a717b4ef7236c4/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:51.152483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:51.152768 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:51.153074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:51.180435 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] MFA Rules not processed for user `34b715a5234240f688a717b4ef7236c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:51.181636 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:51.248335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1197/2359] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:51.259426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:51.261075 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:51.304474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:51.318115 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e208e93-1cc8-40fd-8a99-d41eef7da029 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:51.321727 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:51.322333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:51.322589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:51.322892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:51.332228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:51.333630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dc19-3bda-4868-9be1-4d266745ca42 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:51.343487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1163/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:54.102045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:54.122563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:54.165116 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74f017-dd4e-4ab6-a149-aa7c55279123 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:54.169592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:54.170211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:54.170484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:54.170750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] PATH_INFO: `/v3/users/bdd9a716dd5a4da48a6cabf56e30e0cc` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:54.171961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd9a716dd5a4da48a6cabf56e30e0cc)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:54.173848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:54.368620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd9a716dd5a4da48a6cabf56e30e0cc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:54.375588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd9a716dd5a4da48a6cabf56e30e0cc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:54.376205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd9a716dd5a4da48a6cabf56e30e0cc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:54.444360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd9a716dd5a4da48a6cabf56e30e0cc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:54.445036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd9a716dd5a4da48a6cabf56e30e0cc'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:54.463215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1198/2361] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:29:54 2022] DELETE /identity/v3/users/bdd9a716dd5a4da48a6cabf56e30e0cc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:54.467750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:54.468241 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:54.468352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:54.487198 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:54.488400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:54.546047 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1164/2362] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:29:54.556191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:54.557601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:54.614388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428dfdc-01a5-4a1c-89a6-b502d1e8f34b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:54.617962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:54.618559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:54.618858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:54.619138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:54.628562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:54.629921 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:54.639253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1199/2363] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:55.065707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:55.067195 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:55.079119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0998-9c07-4c46-ad1e-c3c2e82dab1e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:55.083952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:55.084581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:55.084860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:55.085136 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] PATH_INFO: `/v3/projects/c4ad555ef7214224ab0a3eba086cda22` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:55.086348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ad555ef7214224ab0a3eba086cda22)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:55.088293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:55.195888 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ad555ef7214224ab0a3eba086cda22', 'request_id': 'req-42e683cb-7692-4ef1-8eca-f7ee923e869c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.202203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ad555ef7214224ab0a3eba086cda22', 'request_id': 'req-42e683cb-7692-4ef1-8eca-f7ee923e869c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.207824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ad555ef7214224ab0a3eba086cda22', 'request_id': 'req-42e683cb-7692-4ef1-8eca-f7ee923e869c'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.297144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] The token cache is being invalidate because project c4ad555ef7214224ab0a3eba086cda22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:29:55.297434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.298059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.299570 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1165/2364] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:29:55 2022] DELETE /identity/v3/projects/c4ad555ef7214224ab0a3eba086cda22 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:29:55.303566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:55.313255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487172-b987-4e7f-b83f-b1f3c6ad20d5 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:55.352731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:55.368252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:55.368867 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:55.369147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:55.369421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] PATH_INFO: `/v3/users/34b715a5234240f688a717b4ef7236c4` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:55.370512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b715a5234240f688a717b4ef7236c4)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:55.372450 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:55.569070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b715a5234240f688a717b4ef7236c4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.576005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b715a5234240f688a717b4ef7236c4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.576651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b715a5234240f688a717b4ef7236c4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.641212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b715a5234240f688a717b4ef7236c4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.641852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-1adc8e99-6596-4af5-9079-432467bcb525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b715a5234240f688a717b4ef7236c4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:55.660301 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1200/2365] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:29:55 2022] DELETE /identity/v3/users/34b715a5234240f688a717b4ef7236c4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:56.048381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.050842 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:56.064271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e683cb-7692-4ef1-8eca-f7ee923e869c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.121286 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.121910 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.122186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.122457 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] PATH_INFO: `/v3/projects/724495bba7eb4c40962977360d1e47d6` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.123569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=724495bba7eb4c40962977360d1e47d6)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.125548 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.133988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:56.244899 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724495bba7eb4c40962977360d1e47d6', 'request_id': 'req-52d8acab-afb9-4eac-a68a-610ad6c84200'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:56.251052 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724495bba7eb4c40962977360d1e47d6', 'request_id': 'req-52d8acab-afb9-4eac-a68a-610ad6c84200'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:56.257088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724495bba7eb4c40962977360d1e47d6', 'request_id': 'req-52d8acab-afb9-4eac-a68a-610ad6c84200'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:56.323797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] The token cache is being invalidate because project 724495bba7eb4c40962977360d1e47d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:29:56.324089 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:56.324743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-52d8acab-afb9-4eac-a68a-610ad6c84200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:29:56.326253 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1166/2366] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:29:56 2022] DELETE /identity/v3/projects/724495bba7eb4c40962977360d1e47d6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:56.331299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.331777 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.331895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.351569 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:56.352798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:56.410729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1201/2367] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:56.419557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.419838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.420134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.440059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:56.441299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:56.449003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1167/2368] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:56.452996 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.462483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46c5b6-61e6-4700-ad45-01e38e8f3194 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.466943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.467558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.467833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.468097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.468749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:29:56.469049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.470615 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.482465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1202/2369] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:29:56.487246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.496336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558be04-5fb7-4d04-8fc6-53e62d0d26c5 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.500890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.501475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.501753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.502025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.502660 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.504563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.594245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1168/2370] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:29:56 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:29:56.598969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.608926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61622e-71e5-411f-b860-b35226462f50 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.652717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:56.664754 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.665289 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.665568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.665840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.666495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.668432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.785140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1203/2371] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:29:56 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:56.789426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.799478 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58637db-77ca-40ce-964b-e251fb46dfaa admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.803951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.804533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.805764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.805764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.805764 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.807275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.816948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1169/2372] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:56.820821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.831625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e7ff7-631e-4c5d-ba32-6d6472fd4c76 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.836166 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.836749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.837026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.837300 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.837939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.839557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5efc252-ab58-441c-b39c-ba52e6393506 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.850640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1204/2373] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:56.854625 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:56.864321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c4646-e428-4387-af1f-0fc5f6e99064 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:56.868676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:56.869302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.869581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.869850 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] PATH_INFO: `/v3/projects/a52d2bd316884ab58297013a0b20694a/users/0226c9a7ed23473eaa8fe54ecf3dad4c/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.883841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52d2bd316884ab58297013a0b20694a, user_id=0226c9a7ed23473eaa8fe54ecf3dad4c, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:56.886451 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:56.949768 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1170/2374] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:56 2022] PUT /identity/v3/projects/a52d2bd316884ab58297013a0b20694a/users/0226c9a7ed23473eaa8fe54ecf3dad4c/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:56.960194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:56.960483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:56.960755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:56.988310 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] MFA Rules not processed for user `0226c9a7ed23473eaa8fe54ecf3dad4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:56.989507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:57.051930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1205/2375] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:57.056601 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.117834 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd212b3-4a34-44bd-9137-e6e953332f0e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.122516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.123139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.123426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.123704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.124327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.126262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.219978 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1171/2376] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:29:57 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:57.223813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.234388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cec00a-b443-42c4-ad6e-49e248ea495e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.289233 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.289827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.290108 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.290383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.291030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.292950 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.384338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1206/2377] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:29:57 2022] POST /identity/v3/users => generated 302 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:29:57.388287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.399618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04237b1-f8e1-4425-99c1-a69873900dc2 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.404294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.404892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.405175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.405474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.406102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.407731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.418354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1172/2378] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:57.422677 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.432656 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590509f-dc51-4be9-9074-7254e8fd4d3f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.437286 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.437927 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.438203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.438500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] PATH_INFO: `/v3/projects/18b45014b8d849978e730a4625d56ca9/users/3159cd9e3b23495fb910697c5c85b961/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.453432 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b45014b8d849978e730a4625d56ca9, user_id=3159cd9e3b23495fb910697c5c85b961, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.456843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.513131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1207/2379] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:57 2022] PUT /identity/v3/projects/18b45014b8d849978e730a4625d56ca9/users/3159cd9e3b23495fb910697c5c85b961/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:29:57.517079 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.527392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8ab51-b00c-429b-9ace-a5109afc2d38 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.531078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:57.583251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.583849 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.584134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.584407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.585034 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.586618 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccb767-dc0d-4e7d-a33c-7cb1338ec8fc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.596128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1173/2380] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:29:57.600285 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.610345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cadc7e-ca2f-4db5-945d-330da03b8e1f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.615425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.616066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.616349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.616621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] PATH_INFO: `/v3/projects/18b45014b8d849978e730a4625d56ca9/users/3159cd9e3b23495fb910697c5c85b961/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.618471 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b45014b8d849978e730a4625d56ca9, user_id=3159cd9e3b23495fb910697c5c85b961, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.621132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.631832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1208/2381] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:29:57 2022] PUT /identity/v3/projects/18b45014b8d849978e730a4625d56ca9/users/3159cd9e3b23495fb910697c5c85b961/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:29:57.642103 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.642392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.642670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.669907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] MFA Rules not processed for user `3159cd9e3b23495fb910697c5c85b961`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:29:57.671144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:57.738792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1174/2382] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:29:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:29:57.749753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.751498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:57.762804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:29:57.810771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11828b4c-62cf-4b5d-8530-0cfb9d5d7fa4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.814372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.815540 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.815815 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.816085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.825313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.826728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.836076 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1209/2383] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:29:57.881939 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:29:57.883367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:29:57.895298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb64a2-5601-4975-b0bb-68873384e79e None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:29:57.899175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:29:57.899769 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:29:57.900049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:29:57.900325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:29:57.956004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:29:57.957427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e24f6-4537-4367-8fca-e8c0fb9a0fef service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:29:57.966212 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1175/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:29:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:02.686272 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:02.753037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5078559-68c3-4978-80a9-a94c13b98fc6 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:02.757945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:02.758598 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:02.758952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:02.759229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] PATH_INFO: `/v3/users/0226c9a7ed23473eaa8fe54ecf3dad4c` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:02.760455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0226c9a7ed23473eaa8fe54ecf3dad4c)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:02.762395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:02.913383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0226c9a7ed23473eaa8fe54ecf3dad4c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:02.920945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0226c9a7ed23473eaa8fe54ecf3dad4c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:02.921648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0226c9a7ed23473eaa8fe54ecf3dad4c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:02.971726 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0226c9a7ed23473eaa8fe54ecf3dad4c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:02.972307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0226c9a7ed23473eaa8fe54ecf3dad4c'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:02.990495 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1210/2385] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:02 2022] DELETE /identity/v3/users/0226c9a7ed23473eaa8fe54ecf3dad4c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:02.995036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543125db-eebe-439d-86a6-256572c172b0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:02.995540 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543125db-eebe-439d-86a6-256572c172b0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:02.995658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543125db-eebe-439d-86a6-256572c172b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:03.014184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-543125db-eebe-439d-86a6-256572c172b0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:03.015410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-543125db-eebe-439d-86a6-256572c172b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:03.037731 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:03.081399 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1176/2386] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:03.091739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:03.093346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:03.142345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:03.169931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90107df1-ef97-412b-8261-63fb0eacdc55 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:03.174250 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:03.174955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:03.175279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:03.175499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:03.186004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:03.187082 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:03.198069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1211/2387] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:03.704058 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543125db-eebe-439d-86a6-256572c172b0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:03.706272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-543125db-eebe-439d-86a6-256572c172b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:03.721854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543125db-eebe-439d-86a6-256572c172b0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:03.728186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:03.728833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:03.729135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:03.729438 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] PATH_INFO: `/v3/projects/a52d2bd316884ab58297013a0b20694a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:03.730530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52d2bd316884ab58297013a0b20694a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:03.733004 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:03.842498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52d2bd316884ab58297013a0b20694a', 'request_id': 'req-e0575f87-d79c-4b7c-997b-c85697ee1ec4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:03.849615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52d2bd316884ab58297013a0b20694a', 'request_id': 'req-e0575f87-d79c-4b7c-997b-c85697ee1ec4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:03.856260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52d2bd316884ab58297013a0b20694a', 'request_id': 'req-e0575f87-d79c-4b7c-997b-c85697ee1ec4'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:03.937854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] The token cache is being invalidate because project a52d2bd316884ab58297013a0b20694a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:03.938233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:03.938993 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:03.940803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1177/2388] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:03 2022] DELETE /identity/v3/projects/a52d2bd316884ab58297013a0b20694a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:03.945122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:03.955029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17504484-31d4-4acd-a4ff-6be52d1fc16f service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:04.027244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:04.027883 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:04.028160 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:04.028430 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] PATH_INFO: `/v3/users/3159cd9e3b23495fb910697c5c85b961` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:04.029551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3159cd9e3b23495fb910697c5c85b961)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:04.036648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:04.280544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3159cd9e3b23495fb910697c5c85b961'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:04.288595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3159cd9e3b23495fb910697c5c85b961'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:04.289646 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3159cd9e3b23495fb910697c5c85b961'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:04.362466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3159cd9e3b23495fb910697c5c85b961'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:04.363101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-3b908fa4-3d7c-4c78-b2d1-4e680dda2bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3159cd9e3b23495fb910697c5c85b961'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:04.382737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1212/2389] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:03 2022] DELETE /identity/v3/users/3159cd9e3b23495fb910697c5c85b961 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:04.826809 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:04.829173 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:04.857365 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0575f87-d79c-4b7c-997b-c85697ee1ec4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:04.920704 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:04.921363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:04.921644 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:04.921916 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] PATH_INFO: `/v3/projects/18b45014b8d849978e730a4625d56ca9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:04.923083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b45014b8d849978e730a4625d56ca9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:04.925652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.064187 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b45014b8d849978e730a4625d56ca9', 'request_id': 'req-c6b476b4-57fc-4110-a052-63ae7b39f2a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:05.066054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:05.070567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b45014b8d849978e730a4625d56ca9', 'request_id': 'req-c6b476b4-57fc-4110-a052-63ae7b39f2a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:05.075884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b45014b8d849978e730a4625d56ca9', 'request_id': 'req-c6b476b4-57fc-4110-a052-63ae7b39f2a1'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:05.173181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] The token cache is being invalidate because project 18b45014b8d849978e730a4625d56ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:05.173599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:05.174380 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-c6b476b4-57fc-4110-a052-63ae7b39f2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:05.176088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1178/2390] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:04 2022] DELETE /identity/v3/projects/18b45014b8d849978e730a4625d56ca9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:05.181410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.181912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.182050 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.201245 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:05.206512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:05.248913 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:05.261117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1213/2391] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:05.269643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.269930 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.270202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.290518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:05.291823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:05.299581 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1179/2392] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:05.303728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.313269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323a07f-c4a4-462c-bba2-a307b241f703 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.317810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.318398 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.318672 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.318975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.319613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:30:05.320011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.321662 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.333822 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1214/2393] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:30:05.338074 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.347838 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73567ae-4961-4413-8410-6e42a329f02c None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.352525 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.353114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.353396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.353666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.354317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.356249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.436566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1180/2394] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:05.441410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.451381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500f986-b7ce-45b3-add7-1a1bf1428120 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.511782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.512394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.512674 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.512945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.513581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.515553 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.591346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1215/2395] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/users => generated 301 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:30:05.595585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.608533 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4faf6-a661-4e1e-9a0d-cc81c5471392 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.613931 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.614534 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.614839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.615232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.615876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.617497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.628335 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1181/2396] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:05.633630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.644134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69925da1-91f4-430f-a11e-f9d5895277d7 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.649151 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.649787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.650071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.650350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.651026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.652649 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc8c69-b79d-4e8e-af59-a290d68768ca admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.664269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1216/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:05.668264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:05.680208 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ac17d-2959-43d2-841a-ad9e8b78f2c7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:05.685078 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:05.685732 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.686011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.686281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] PATH_INFO: `/v3/projects/a1be550efb70419f9b8bf797456173ab/users/4815e2dad9cb44049c69bc461ebcf220/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.701385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1be550efb70419f9b8bf797456173ab, user_id=4815e2dad9cb44049c69bc461ebcf220, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:05.704073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:05.825799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1182/2398] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:05 2022] PUT /identity/v3/projects/a1be550efb70419f9b8bf797456173ab/users/4815e2dad9cb44049c69bc461ebcf220/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:05.837034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:05.837293 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:05.837940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:05.868251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] MFA Rules not processed for user `4815e2dad9cb44049c69bc461ebcf220`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:05.869684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:05.936587 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1217/2399] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:05.941233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.005995 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916850b-536d-4ff9-aa36-24e8b96715af admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.010841 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.011530 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.011813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.012084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.012719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.014603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.124057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1183/2400] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:05 2022] POST /identity/v3/projects => generated 399 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:06.128446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.138946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6a7c36-eafa-4d33-a809-4422fc94cf67 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.183916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:06.196533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.197137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.197416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.197686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.198360 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.200315 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.310379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1218/2401] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:06 2022] POST /identity/v3/users => generated 306 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:06.314092 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.324316 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1339074-35d8-4c3d-9ad1-e4d5d8af553f admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.328890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.329476 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.329755 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.330028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.330656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.332291 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.348114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1184/2402] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:06.351003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.360971 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067de0a-dcb5-4132-b20c-0157dd00af17 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.366428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.367211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.367458 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.367731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] PATH_INFO: `/v3/projects/885f59db4c314aa4a2a0c8cbb377baef/users/69ecaf264c4940eea8ba9544210425da/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.384030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=885f59db4c314aa4a2a0c8cbb377baef, user_id=69ecaf264c4940eea8ba9544210425da, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.386685 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.457297 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1219/2403] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:06 2022] PUT /identity/v3/projects/885f59db4c314aa4a2a0c8cbb377baef/users/69ecaf264c4940eea8ba9544210425da/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:06.461336 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.472537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb2a1d-67f4-4eeb-b471-ca66ded79446 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.526847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:06.537022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.537617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.537893 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.538162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.538784 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.540386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811d53f-34e2-4447-99ce-af361755be7d admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.550008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1185/2404] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:06.554147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.564581 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebafc-5f01-4e19-b5a6-ff5f3552c1f5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.569550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.570247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.570583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.570804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] PATH_INFO: `/v3/projects/885f59db4c314aa4a2a0c8cbb377baef/users/69ecaf264c4940eea8ba9544210425da/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.572848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] RBAC: Authorizing `identity:create_grant(project_id=885f59db4c314aa4a2a0c8cbb377baef, user_id=69ecaf264c4940eea8ba9544210425da, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.575597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.587088 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1220/2405] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:06 2022] PUT /identity/v3/projects/885f59db4c314aa4a2a0c8cbb377baef/users/69ecaf264c4940eea8ba9544210425da/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:06.655651 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.655872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.656114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.683962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] MFA Rules not processed for user `69ecaf264c4940eea8ba9544210425da`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:06.685176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:06.754183 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1186/2406] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:06.765212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:06.766993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:06.825896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a78477-0fef-4dd3-b940-5596b74a552f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:06.829542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:06.830170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:06.830965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:06.833053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:06.840294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:06.841712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:06.851423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1221/2407] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:07.088376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:07.098275 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c980-ebd0-4355-9488-e92f2a83b93d None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:07.101805 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:07.102344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:07.102675 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:07.102979 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:07.163160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:07.164707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86104489-53e4-4869-839b-8578676bc47c service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:07.175119 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1187/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:07 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:13.474075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:13.571869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b74ef-e5ee-4c27-b827-b980e9268754 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:13.576686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:13.583328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:13.583328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:13.583328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] PATH_INFO: `/v3/users/4815e2dad9cb44049c69bc461ebcf220` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:13.583328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4815e2dad9cb44049c69bc461ebcf220)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:13.583328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:13.796189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:13.798745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4815e2dad9cb44049c69bc461ebcf220'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:13.807120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4815e2dad9cb44049c69bc461ebcf220'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:13.808102 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4815e2dad9cb44049c69bc461ebcf220'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:13.899832 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4815e2dad9cb44049c69bc461ebcf220'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:13.900456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4815e2dad9cb44049c69bc461ebcf220'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:13.922809 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1222/2409] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:13 2022] DELETE /identity/v3/users/4815e2dad9cb44049c69bc461ebcf220 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:13.927100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:13.931149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:13.931149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:13.959641 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:13.964696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:14.044168 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1188/2410] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:30:14.052795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:14.054232 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:14.120339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91536cd6-2590-4b35-91ba-9aeb88883806 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:14.124585 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:14.125194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:14.125469 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:14.125740 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:14.135366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:14.136818 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:14.148252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1223/2411] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:14.574915 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:14.576857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:14.621246 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8932-d55b-42e3-94ea-5aed2a566cc6 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:14.626297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:14.628719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:14.629010 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:14.629468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] PATH_INFO: `/v3/projects/a1be550efb70419f9b8bf797456173ab` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:14.630736 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1be550efb70419f9b8bf797456173ab)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:14.635022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:14.644906 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:14.876922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1be550efb70419f9b8bf797456173ab', 'request_id': 'req-31b8b719-6c92-4a8f-8157-5742d3410897'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:14.884245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1be550efb70419f9b8bf797456173ab', 'request_id': 'req-31b8b719-6c92-4a8f-8157-5742d3410897'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:14.889980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1be550efb70419f9b8bf797456173ab', 'request_id': 'req-31b8b719-6c92-4a8f-8157-5742d3410897'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:14.977063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] The token cache is being invalidate because project a1be550efb70419f9b8bf797456173ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:14.977063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:14.977657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:14.980031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1189/2412] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:14 2022] DELETE /identity/v3/projects/a1be550efb70419f9b8bf797456173ab => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:14.985146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:14.995463 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebc867-9136-4f03-af04-e0652ca09e3e service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:15.065798 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:15.066492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:15.066771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:15.067131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] PATH_INFO: `/v3/users/69ecaf264c4940eea8ba9544210425da` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:15.068283 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ecaf264c4940eea8ba9544210425da)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:15.070635 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:15.276848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ecaf264c4940eea8ba9544210425da'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:15.284563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ecaf264c4940eea8ba9544210425da'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:15.285238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ecaf264c4940eea8ba9544210425da'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:15.367639 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ecaf264c4940eea8ba9544210425da'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:15.368264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-b2ce6932-6e06-46d9-bfd0-86b03d34e07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ecaf264c4940eea8ba9544210425da'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:15.373525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:15.392363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1224/2413] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:14 2022] DELETE /identity/v3/users/69ecaf264c4940eea8ba9544210425da => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:15.883510 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:15.884676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:15.898630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b719-6c92-4a8f-8157-5742d3410897 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.014770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.015584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.015835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.016107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] PATH_INFO: `/v3/projects/885f59db4c314aa4a2a0c8cbb377baef` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.017743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=885f59db4c314aa4a2a0c8cbb377baef)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.020806 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.134729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '885f59db4c314aa4a2a0c8cbb377baef', 'request_id': 'req-612fd88f-039b-4c54-8bfa-d1f4285ee087'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:16.141407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '885f59db4c314aa4a2a0c8cbb377baef', 'request_id': 'req-612fd88f-039b-4c54-8bfa-d1f4285ee087'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:16.147131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '885f59db4c314aa4a2a0c8cbb377baef', 'request_id': 'req-612fd88f-039b-4c54-8bfa-d1f4285ee087'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:16.222730 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] The token cache is being invalidate because project 885f59db4c314aa4a2a0c8cbb377baef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:16.223055 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:16.228273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-612fd88f-039b-4c54-8bfa-d1f4285ee087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:16.228273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1190/2414] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:15 2022] DELETE /identity/v3/projects/885f59db4c314aa4a2a0c8cbb377baef => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:16.231720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.231720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.231720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.253440 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:16.254747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:16.336358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1225/2415] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:16.344186 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.344477 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.344751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.366219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:16.367487 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:16.375619 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1191/2416] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:16.380005 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.391601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dda52f-007a-48b2-b58a-068a63088011 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.397515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.398128 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.398464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.399077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.399849 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:30:16.400138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.402156 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.415548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1226/2417] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:30:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:30:16.419706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.429015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e5318-5ead-4975-bebd-3213b1748d03 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.433686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.434292 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.434569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.434855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.435807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.437405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.456864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:16.530752 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1192/2418] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:16.535473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.545896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21629f4-a0ac-44ae-8e10-4f065032cef2 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.604654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.605260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.605542 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.605817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.606488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.608448 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.694443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1227/2419] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:16.698799 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.709669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6deed6-9690-4962-9197-75cfa99d0965 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.714201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.714795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.715099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.715375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.716014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.717670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.730278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1193/2420] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:16.734575 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.751046 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3222b-646a-4f5b-9a1b-869c5ff3c5ad admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.756466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.757075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.757356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.757630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.758274 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.760077 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72bf39-3bd0-47b5-a46f-5aa24f65a57e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.771872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1228/2421] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:16.776794 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:16.786578 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2799-ced9-4d89-8fd2-01ab4f2d79ed admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:16.793587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:16.793587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.793587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.793587 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] PATH_INFO: `/v3/projects/7f2a7f5595c3403eb9f78a1b8856c58f/users/f53e74cace164c9eb6eea1ab6567e6e0/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.807967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2a7f5595c3403eb9f78a1b8856c58f, user_id=f53e74cace164c9eb6eea1ab6567e6e0, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:16.810595 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:16.883240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1194/2422] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:16 2022] PUT /identity/v3/projects/7f2a7f5595c3403eb9f78a1b8856c58f/users/f53e74cace164c9eb6eea1ab6567e6e0/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:16.900684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:16.900966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:16.901237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:16.928072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] MFA Rules not processed for user `f53e74cace164c9eb6eea1ab6567e6e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:16.929260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:16.969781 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:16.993637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1229/2423] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:16.998332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.066133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e37fa-7427-4921-a6e3-ce1759c5ce43 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.070981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.072270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.072270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.072270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.072918 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.074848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.183019 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1195/2424] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:16 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:17.185559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.197008 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4c3c6-d1b3-488f-b2ae-8f00707b7f09 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.253162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.253803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.253916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.254149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.254773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.256729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.354912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1230/2425] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:17 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:17.358321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.370348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540fcce-159e-4222-a507-7c61e162cadf admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.375179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.375782 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.376057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.376328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.376959 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.379128 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.389863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1196/2426] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:17.394548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.404972 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e1334-223e-4f23-93ec-5db24de0408b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.409774 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.410673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.410673 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.410945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.411582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.413197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d1b9c2-1cf8-45ae-a081-72f671246ec8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.423427 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1231/2427] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:17.427567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.437654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff05b4-408d-4bf8-81c4-ba8a98e484c8 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.442919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.443567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.443848 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.444129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] PATH_INFO: `/v3/projects/bfa5a3eaeabf459da037715fe74d95c8/users/dfd5912194c74ef1a6c7a303aee3e99f/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.459379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa5a3eaeabf459da037715fe74d95c8, user_id=dfd5912194c74ef1a6c7a303aee3e99f, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.462087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.548258 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1197/2428] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:17 2022] PUT /identity/v3/projects/bfa5a3eaeabf459da037715fe74d95c8/users/dfd5912194c74ef1a6c7a303aee3e99f/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:17.559630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.559936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.560189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.589390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] MFA Rules not processed for user `dfd5912194c74ef1a6c7a303aee3e99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:17.590621 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:17.656202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1232/2429] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:17.660711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.725163 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee2c8-8577-4bd5-a6ca-cc9c593af3c1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.729917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.731954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.731954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.731954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.731954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.733735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:17.751307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:17.866967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1198/2430] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:17 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:17.871251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:17.882936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f63ae0-d4aa-4c9d-824c-15e37df7f7c6 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:17.927109 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:17.938641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:17.939295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:17.942964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:17.942964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:17.942964 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:17.943268 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.058851 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1233/2431] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:17 2022] POST /identity/v3/users => generated 295 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:18.062785 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.073744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364caf-9819-4e4c-be66-1e6c359b2938 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.078347 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.078973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.079844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.079844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.080182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.081780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.092773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1199/2432] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:18.097842 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.108902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad85fd-03da-4aca-91af-16dc2e863e9c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.114334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.115053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.121762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.121762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] PATH_INFO: `/v3/projects/5a0f467ad4fc404eb5710985a487fabf/users/d8094e0648f84077a72b6bcadd47c75a/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.130618 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0f467ad4fc404eb5710985a487fabf, user_id=d8094e0648f84077a72b6bcadd47c75a, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.133303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.196999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1234/2433] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:18 2022] PUT /identity/v3/projects/5a0f467ad4fc404eb5710985a487fabf/users/d8094e0648f84077a72b6bcadd47c75a/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:18.200925 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.211396 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4daaa0-0018-4d9e-9d49-bc34afaae41c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.285120 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.286090 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.286642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.287135 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.288132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.290750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cd416-00b0-4edc-8dad-88ff9b3c2f6b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.304953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1200/2434] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:18.310444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.321492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162c62c-5a3e-4e1e-a7a2-79cdc057865a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.327350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.332132 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.332444 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.332722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] PATH_INFO: `/v3/projects/5a0f467ad4fc404eb5710985a487fabf/users/d8094e0648f84077a72b6bcadd47c75a/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.335372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0f467ad4fc404eb5710985a487fabf, user_id=d8094e0648f84077a72b6bcadd47c75a, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.338101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.349841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1235/2435] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:18 2022] PUT /identity/v3/projects/5a0f467ad4fc404eb5710985a487fabf/users/d8094e0648f84077a72b6bcadd47c75a/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:18.360164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.360456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.360735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.395174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] MFA Rules not processed for user `d8094e0648f84077a72b6bcadd47c75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:18.396344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:18.472508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1201/2436] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:18.483825 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.485531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:18.559666 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd47d9b-ab9a-4195-a005-fcd5724744b1 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.563311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.563917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.564188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.564456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.573291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.574702 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.584998 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1236/2437] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:18.656617 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.658032 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:18.712161 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:18.713700 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:18.717855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1cd9b-5086-4a85-9faa-b5473acf1345 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.722072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.722686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.723003 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.723307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.725081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:18.725792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16645d8-a445-4f74-84b4-353a520ffb9d service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:18.730562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:18.731219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:18.731503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:18.731771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:18.746407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:18.792281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.793706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.795265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:18.796718 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:18.804376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1202/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:18.806962 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1237/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:30:22.599385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:22.600942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:22.681622 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9c65d-74dd-4296-a023-0a81bf32ae26 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:22.686696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:22.687332 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:22.687646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:22.687920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:22.689657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:22.701498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:22.703101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd2167-b331-45f4-85e2-8050420bce8e service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:22.713662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1203/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:24.248848 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:24.265657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549df941-9e72-4a81-b269-b9f86446fbc1 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:24.279616 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:24.280270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:24.280550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:24.280821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] PATH_INFO: `/v3/users/f53e74cace164c9eb6eea1ab6567e6e0` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:24.282001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53e74cace164c9eb6eea1ab6567e6e0)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:24.284180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:24.496555 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:24.505150 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53e74cace164c9eb6eea1ab6567e6e0'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:24.514323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53e74cace164c9eb6eea1ab6567e6e0'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:24.515364 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53e74cace164c9eb6eea1ab6567e6e0'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:24.579923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53e74cace164c9eb6eea1ab6567e6e0'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:24.580993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53e74cace164c9eb6eea1ab6567e6e0'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:24.603505 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1238/2441] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:24 2022] DELETE /identity/v3/users/f53e74cace164c9eb6eea1ab6567e6e0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:24.608108 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:24.609176 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:24.609686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:24.639067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:24.640507 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:24.778913 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1204/2442] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:24.790029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:24.792468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:24.901014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace95f87-1d53-4cf8-8fab-0936f37ef5d4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:24.904780 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:24.905508 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:24.905854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:24.906201 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:24.918531 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:24.921237 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:24.939762 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1239/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:30:25.401923 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:25.403854 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:25.420804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6882b0-cab2-422a-8064-f5ffcd75e3a7 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:25.426589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:25.427391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:25.427791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:25.428244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] PATH_INFO: `/v3/projects/7f2a7f5595c3403eb9f78a1b8856c58f` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:25.429405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2a7f5595c3403eb9f78a1b8856c58f)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:25.431698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:25.527434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:25.533702 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2a7f5595c3403eb9f78a1b8856c58f', 'request_id': 'req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.540225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2a7f5595c3403eb9f78a1b8856c58f', 'request_id': 'req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.545765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2a7f5595c3403eb9f78a1b8856c58f', 'request_id': 'req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.624648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] The token cache is being invalidate because project 7f2a7f5595c3403eb9f78a1b8856c58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:25.625270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.625520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.627134 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1205/2444] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:25 2022] DELETE /identity/v3/projects/7f2a7f5595c3403eb9f78a1b8856c58f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:25.631197 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:25.640924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057760d-0914-43f3-ad98-6063150709d8 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:25.703517 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:25.704541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:25.705052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:25.705699 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] PATH_INFO: `/v3/users/dfd5912194c74ef1a6c7a303aee3e99f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:25.707872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfd5912194c74ef1a6c7a303aee3e99f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:25.710079 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:25.906745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfd5912194c74ef1a6c7a303aee3e99f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.917136 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfd5912194c74ef1a6c7a303aee3e99f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.918066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfd5912194c74ef1a6c7a303aee3e99f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:25.920978 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:26.066196 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfd5912194c74ef1a6c7a303aee3e99f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.067956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfd5912194c74ef1a6c7a303aee3e99f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.087392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1240/2445] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:25 2022] DELETE /identity/v3/users/dfd5912194c74ef1a6c7a303aee3e99f => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:26.596955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:26.596955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:26.608718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4c67-0abb-407e-8fb7-0b94a0ad31d6 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:26.683437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:26.684073 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:26.684623 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:26.684908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] PATH_INFO: `/v3/projects/bfa5a3eaeabf459da037715fe74d95c8` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:26.686114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa5a3eaeabf459da037715fe74d95c8)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:26.688038 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:26.790023 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:26.791918 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:26.808114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd18bd-3791-4071-8ef2-1e263786a7da admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:26.811863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa5a3eaeabf459da037715fe74d95c8', 'request_id': 'req-faf4af7d-d772-4fb9-9925-57c901e831bf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.818490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa5a3eaeabf459da037715fe74d95c8', 'request_id': 'req-faf4af7d-d772-4fb9-9925-57c901e831bf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.824325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa5a3eaeabf459da037715fe74d95c8', 'request_id': 'req-faf4af7d-d772-4fb9-9925-57c901e831bf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.868582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:26.869305 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:26.869586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:26.869860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] PATH_INFO: `/v3/users/1f737b4d0531425cbc803fcd15a3ea1f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:26.883333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f737b4d0531425cbc803fcd15a3ea1f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:26.885242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:26.916054 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] The token cache is being invalidate because project bfa5a3eaeabf459da037715fe74d95c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:26.916329 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.916946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:26.918419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1206/2446] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:26 2022] DELETE /identity/v3/projects/bfa5a3eaeabf459da037715fe74d95c8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:26.922514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:26.923862 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:26.936306 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4af7d-d772-4fb9-9925-57c901e831bf admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:26.941323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:26.941942 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:26.942221 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:26.942494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] PATH_INFO: `/v3/users/d8094e0648f84077a72b6bcadd47c75a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:26.943676 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8094e0648f84077a72b6bcadd47c75a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:26.945571 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:27.097597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f737b4d0531425cbc803fcd15a3ea1f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.104482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f737b4d0531425cbc803fcd15a3ea1f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.105149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f737b4d0531425cbc803fcd15a3ea1f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.154967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8094e0648f84077a72b6bcadd47c75a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.160434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:27.162466 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8094e0648f84077a72b6bcadd47c75a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.163145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8094e0648f84077a72b6bcadd47c75a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.191190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f737b4d0531425cbc803fcd15a3ea1f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.191813 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f737b4d0531425cbc803fcd15a3ea1f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.210321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1241/2447] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:26 2022] DELETE /identity/v3/users/1f737b4d0531425cbc803fcd15a3ea1f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:27.274455 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8094e0648f84077a72b6bcadd47c75a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.275232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8094e0648f84077a72b6bcadd47c75a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.293662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1207/2448] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:26 2022] DELETE /identity/v3/users/d8094e0648f84077a72b6bcadd47c75a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:27.458513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:27.460013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:27.511296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:27.531213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116beff-ae6b-4538-9e59-59f65f3621bd admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:27.594014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:27.594939 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:27.595180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:27.595453 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] PATH_INFO: `/v3/projects/e378bfab9d91492e81320139fed867c6` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:27.606493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e378bfab9d91492e81320139fed867c6)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:27.608438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:27.720072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e378bfab9d91492e81320139fed867c6', 'request_id': 'req-9e1a426f-da9c-48fa-89f7-137e70ae4a26'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.727031 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e378bfab9d91492e81320139fed867c6', 'request_id': 'req-9e1a426f-da9c-48fa-89f7-137e70ae4a26'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.733582 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e378bfab9d91492e81320139fed867c6', 'request_id': 'req-9e1a426f-da9c-48fa-89f7-137e70ae4a26'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.739277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:27.740750 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:27.758227 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c830c2-f46b-4e02-9268-63b01ec037bc admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:27.808067 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] The token cache is being invalidate because project e378bfab9d91492e81320139fed867c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:27.808604 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.809174 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.811032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1242/2449] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:27 2022] DELETE /identity/v3/projects/e378bfab9d91492e81320139fed867c6 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:27.815551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:27.816644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:27.831325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:27.831987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:27.832265 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:27.832542 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] PATH_INFO: `/v3/projects/5a0f467ad4fc404eb5710985a487fabf` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:27.833699 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a0f467ad4fc404eb5710985a487fabf)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:27.835664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:27.891744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a426f-da9c-48fa-89f7-137e70ae4a26 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:27.897101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:27.897734 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:27.898011 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:27.898282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] PATH_INFO: `/v3/users/28e0de7076904034a98885c8ebeddce3` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:27.913905 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e0de7076904034a98885c8ebeddce3)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:27.915952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:27.918175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:27.948562 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a0f467ad4fc404eb5710985a487fabf', 'request_id': 'req-5a0a2845-5ebd-4d53-8921-f385f04f6000'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.954662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a0f467ad4fc404eb5710985a487fabf', 'request_id': 'req-5a0a2845-5ebd-4d53-8921-f385f04f6000'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:27.960998 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a0f467ad4fc404eb5710985a487fabf', 'request_id': 'req-5a0a2845-5ebd-4d53-8921-f385f04f6000'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.041337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] The token cache is being invalidate because project 5a0f467ad4fc404eb5710985a487fabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:28.041535 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.042196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-5a0a2845-5ebd-4d53-8921-f385f04f6000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.043872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1208/2450] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:27 2022] DELETE /identity/v3/projects/5a0f467ad4fc404eb5710985a487fabf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:28.049681 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9140145-f4ad-4fdb-bc1d-619c15f98a39 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.049943 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9140145-f4ad-4fdb-bc1d-619c15f98a39 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.050220 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9140145-f4ad-4fdb-bc1d-619c15f98a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.070131 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b9140145-f4ad-4fdb-bc1d-619c15f98a39 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:28.071378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b9140145-f4ad-4fdb-bc1d-619c15f98a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:28.120255 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e0de7076904034a98885c8ebeddce3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.127236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e0de7076904034a98885c8ebeddce3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.127928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e0de7076904034a98885c8ebeddce3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.131105 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1209/2451] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:30:28.140701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.141059 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.141403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.154091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:28.166967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:28.168179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:28.176552 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1210/2452] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:28.180637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.189447 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347dff3-21c0-46e7-981a-26cdd0a4c4f9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.194614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.195215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.195492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.195765 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.196400 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:30:28.196703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.198352 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.210302 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1211/2453] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:30:28.214550 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.219231 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e0de7076904034a98885c8ebeddce3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.219878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e0de7076904034a98885c8ebeddce3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.224433 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6eaaf-5308-4043-b1ab-97dfd3c35fa7 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.229392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.229981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.230264 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.230541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.231313 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.233046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.239465 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1243/2454] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:27 2022] DELETE /identity/v3/users/28e0de7076904034a98885c8ebeddce3 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:28.316024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1212/2455] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:28 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:30:28.319966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.321820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:28.389295 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76edfb-56fd-4ed6-864c-ada79f08cd7c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.448346 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.448961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.449265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.449525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.450170 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.452193 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.483767 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.485577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:28.501929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053e81-6000-4a6e-a783-e4bdbc3f5335 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.508219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.508946 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.509202 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.509637 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] PATH_INFO: `/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.520770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ed68e985764bc5b139aa5a9de6bd01)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.522748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.557388 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1244/2456] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:28 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:30:28.561311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.571583 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1d96d-b123-44ab-be02-4cdba20e60fb admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.576103 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.576676 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.576961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.577235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.577893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.579522 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.589365 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1245/2457] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:28.594855 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.604627 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030c32-8a90-4e19-b21f-809d59a722a4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.609235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.609835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.610122 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.610399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.611042 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.615202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.615202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:28.628048 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1246/2458] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:28.628989 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.633996 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ed68e985764bc5b139aa5a9de6bd01', 'request_id': 'req-dc38a951-8440-4a5a-9322-21a689325994'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.638586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543080c8-973a-498e-8a71-9f69c8aa1fd3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:28.642285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ed68e985764bc5b139aa5a9de6bd01', 'request_id': 'req-dc38a951-8440-4a5a-9322-21a689325994'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.648310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ed68e985764bc5b139aa5a9de6bd01', 'request_id': 'req-dc38a951-8440-4a5a-9322-21a689325994'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.696392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:28.697045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.697328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.697606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] PATH_INFO: `/v3/projects/9e07fd88c62a4176bdc905b7da64f114/users/ce9062e473bb44a78888e3328c3db767/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.711976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e07fd88c62a4176bdc905b7da64f114, user_id=ce9062e473bb44a78888e3328c3db767, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:28.714611 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:28.742808 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] The token cache is being invalidate because project a6ed68e985764bc5b139aa5a9de6bd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:28.743353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.743763 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-dc38a951-8440-4a5a-9322-21a689325994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:28.745311 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1213/2459] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:28 2022] DELETE /identity/v3/projects/a6ed68e985764bc5b139aa5a9de6bd01 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:28.823629 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1247/2460] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:28 2022] PUT /identity/v3/projects/9e07fd88c62a4176bdc905b7da64f114/users/ce9062e473bb44a78888e3328c3db767/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:30:28.835614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:28.835908 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:28.836185 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:28.869886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] MFA Rules not processed for user `ce9062e473bb44a78888e3328c3db767`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:28.871175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:28.942338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1214/2461] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:28.962118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:28.964129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:29.025854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f25fe-6092-4f47-a085-9d1320da415b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:29.029763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:29.030374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:29.030657 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:29.030973 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:29.039839 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:29.041263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a5562-c9a6-470c-b4e0-177d2ddf24a7 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:29.049923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1248/2462] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:38.938143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=80,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:39.045940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:40.064628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:40.066036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:40.131789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4ba3a-8026-4af2-9bb4-d04d4914bc66 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:40.136289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:40.136911 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:40.137191 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:40.137463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] PATH_INFO: `/v3/users/ce9062e473bb44a78888e3328c3db767` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:40.138576 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9062e473bb44a78888e3328c3db767)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:40.140633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:40.318427 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9062e473bb44a78888e3328c3db767'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:40.325207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9062e473bb44a78888e3328c3db767'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:40.325905 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9062e473bb44a78888e3328c3db767'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:40.401945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9062e473bb44a78888e3328c3db767'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:40.402563 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9062e473bb44a78888e3328c3db767'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:40.419783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1215/2463] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:30:40 2022] DELETE /identity/v3/users/ce9062e473bb44a78888e3328c3db767 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:40.424578 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:40.425075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:40.425188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:40.444229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:40.445415 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:40.502131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1249/2464] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:40.512342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:40.513745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:40.570468 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff32a3-88ac-4ae9-82c0-6943340ce413 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:40.574031 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:40.574665 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:40.575184 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:40.575481 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:40.585148 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:40.586559 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d69e1-7552-4f76-a718-1435c7d39124 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:40.596428 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1216/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:41.009263 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.010721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:41.025000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b63390-2461-4d66-819b-8e92bc9016bc None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.030299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.031123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.031402 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.031827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] PATH_INFO: `/v3/projects/9e07fd88c62a4176bdc905b7da64f114` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.032873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e07fd88c62a4176bdc905b7da64f114)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.034688 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.133557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e07fd88c62a4176bdc905b7da64f114', 'request_id': 'req-958bf94c-52b1-4098-9762-e56c7b65b6ac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:41.139896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e07fd88c62a4176bdc905b7da64f114', 'request_id': 'req-958bf94c-52b1-4098-9762-e56c7b65b6ac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:41.145473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e07fd88c62a4176bdc905b7da64f114', 'request_id': 'req-958bf94c-52b1-4098-9762-e56c7b65b6ac'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:41.229853 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] The token cache is being invalidate because project 9e07fd88c62a4176bdc905b7da64f114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:30:41.230142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:41.230757 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-958bf94c-52b1-4098-9762-e56c7b65b6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:30:41.232423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1250/2466] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:30:41 2022] DELETE /identity/v3/projects/9e07fd88c62a4176bdc905b7da64f114 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:41.237278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.237762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.237863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.258569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:41.259798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:41.276320 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:41.318309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1217/2467] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:41.326095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.326378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.326648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.346960 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:41.348776 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:41.356655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1251/2468] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:41.360694 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.370348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42e62f-fcd6-4c33-986f-0fd6e14b3e24 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.374791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.375403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.375678 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.375944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.376583 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:30:41.376880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.378449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.389630 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1218/2469] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:30:41.393749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.403554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095b39f-6a72-455a-bc93-8b8c2ad8b119 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.408209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.408819 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.409090 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.409358 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.409974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.412503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.503422 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1252/2470] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:30:41 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:41.507473 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.517372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2af559-c4b2-483a-b855-6e6692ffe574 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.573114 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.573707 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.574101 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.574397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.575041 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.576980 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.679709 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1219/2471] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:30:41 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:30:41.683924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.694736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72e140-e9a2-4209-ace5-9fee9ceb64ea admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.699409 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.700029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.700306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.700577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.701337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.702948 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.713395 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1253/2472] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:30:41.717175 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.727098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b44af-cc98-47bb-b65d-51d40340bcd2 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.731733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.732323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.732605 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.732879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.733512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.735122 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cd584-d4d8-4553-812c-3c6c2cda558b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.745977 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1220/2473] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 26 11:30:41.749956 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.760651 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb9aa5-d8c6-4674-b7fa-3ad9d1624d50 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:41.765212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:41.765854 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.766134 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.766408 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] PATH_INFO: `/v3/projects/044799c63ce94aa39ea143e1633e2002/users/18665db956ab4da28230c4c2bfe8ffa7/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.783135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044799c63ce94aa39ea143e1633e2002, user_id=18665db956ab4da28230c4c2bfe8ffa7, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:41.785778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:41.868349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1254/2474] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:30:41 2022] PUT /identity/v3/projects/044799c63ce94aa39ea143e1633e2002/users/18665db956ab4da28230c4c2bfe8ffa7/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:30:41.873319 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809334aa-323b-4dc7-8c37-47ea4c32b52d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:41.873791 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809334aa-323b-4dc7-8c37-47ea4c32b52d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:41.874067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809334aa-323b-4dc7-8c37-47ea4c32b52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:41.893843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-809334aa-323b-4dc7-8c37-47ea4c32b52d None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:41.895751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-809334aa-323b-4dc7-8c37-47ea4c32b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:41.954378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1221/2475] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:41.964663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:41.966307 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:41.987856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:42.022789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f170bd90-8c55-47fa-928f-9bd151e86f18 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:42.026640 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:42.027264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:42.027541 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:42.027810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:42.036566 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:42.037965 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:42.047203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1255/2476] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:43.384813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cabd-5e6d-4cf4-8df4-dddf44d71b83 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:43.385391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cabd-5e6d-4cf4-8df4-dddf44d71b83 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:43.385391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772cabd-5e6d-4cf4-8df4-dddf44d71b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:43.415560 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c772cabd-5e6d-4cf4-8df4-dddf44d71b83 None None] MFA Rules not processed for user `18665db956ab4da28230c4c2bfe8ffa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:43.416813 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c772cabd-5e6d-4cf4-8df4-dddf44d71b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:43.434729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:43.485371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1222/2477] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:43.496028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:30:43.497558 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:43.558379 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3fb5f-5110-4f48-be12-438893798b80 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:30:43.561995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:30:43.562600 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:43.563013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:43.563287 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:43.571995 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:30:43.573500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:30:43.582872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1256/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:30:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:30:45.258388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a2b1-502e-42ea-9a15-4f729a413f6d None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:30:45.258971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a2b1-502e-42ea-9a15-4f729a413f6d None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:30:45.258971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a2b1-502e-42ea-9a15-4f729a413f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:30:45.278565 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-5772a2b1-502e-42ea-9a15-4f729a413f6d None None] MFA Rules not processed for user `18665db956ab4da28230c4c2bfe8ffa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:30:45.279804 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-5772a2b1-502e-42ea-9a15-4f729a413f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:30:45.287244 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1223/2479] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:30:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:30:53.578328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=43 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:30:55.272909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=22 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:31:51.998872 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:31:52.000273 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:52.070626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01028d7-3be1-4158-889a-50ebaacf7528 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:31:52.076064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:31:52.076669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:31:52.076949 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:31:52.077222 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:31:52.078986 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:52.090597 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:31:52.092190 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:31:52.102588 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1257/2480] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:31:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:31:54.081166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:31:54.081628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:31:54.081733 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:31:54.131229 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] MFA Rules not processed for user `016e7bb933254672b760d32da32eb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:31:54.132425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:54.195738 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1224/2481] 10.222.0.42 () {58 vars in 967 bytes} [Wed Oct 26 11:31:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:31:54.235869 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:31:54.237946 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:54.326004 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eb504-beb7-402b-8aa9-670b49616cdd service placement] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:31:54.330831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd256290285d44aafb8062ec2059b16ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:31:54.331544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:31:54.331821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:31:54.332089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:31:54.339850 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:31:54.341397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:31:54.349705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1258/2482] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Oct 26 11:31:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 26 11:31:59.469372 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:31:59.476561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:59.487635 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334d464-ab12-4dff-a85e-f5bf02a99ca0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:31:59.492504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:31:59.493102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:31:59.493381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:31:59.493648 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:31:59.495504 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:31:59.507713 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:31:59.509284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:31:59.519430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1225/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:31:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:32:03.308140 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:32:03.309681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:03.329455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90aa54-b3e2-4863-bab2-ec1d54c02e97 service swift] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:32:03.329455 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:32:03.329977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:03.330260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:03.330544 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] PATH_INFO: `/v3/users/e7088cf9374c4e7d8e60c34a782fb42f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:03.331723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7088cf9374c4e7d8e60c34a782fb42f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:32:03.333628 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:32:03.491810 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7088cf9374c4e7d8e60c34a782fb42f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:03.497038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:32:03.498983 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7088cf9374c4e7d8e60c34a782fb42f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:03.499642 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7088cf9374c4e7d8e60c34a782fb42f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:03.560623 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7088cf9374c4e7d8e60c34a782fb42f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:03.561179 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7088cf9374c4e7d8e60c34a782fb42f'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:03.579138 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1259/2484] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:32:03 2022] DELETE /identity/v3/users/e7088cf9374c4e7d8e60c34a782fb42f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:32:03.835393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:32:03.846887 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bccbd28-503e-4c1c-bf4a-f99cd68b6b45 service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:32:03.921424 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:32:03.922083 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:03.922368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:03.922703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] PATH_INFO: `/v3/projects/a975701f3c69481b874e106eb5d4edcd` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:03.923823 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a975701f3c69481b874e106eb5d4edcd)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:32:03.926262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:32:04.055386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a975701f3c69481b874e106eb5d4edcd', 'request_id': 'req-b1542939-d6de-404f-bb8d-be55bbdbb00a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.061585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a975701f3c69481b874e106eb5d4edcd', 'request_id': 'req-b1542939-d6de-404f-bb8d-be55bbdbb00a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.066882 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:32:04.068174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a975701f3c69481b874e106eb5d4edcd', 'request_id': 'req-b1542939-d6de-404f-bb8d-be55bbdbb00a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.150771 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] The token cache is being invalidate because project a975701f3c69481b874e106eb5d4edcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:32:04.151098 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.151779 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.153274 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1226/2485] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:32:03 2022] DELETE /identity/v3/projects/a975701f3c69481b874e106eb5d4edcd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:32:04.157332 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:32:04.158925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:04.172390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fbd1a-3347-46ac-bb62-c298aaaa325e admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:32:04.228421 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:32:04.229097 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:04.229383 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:04.229660 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] PATH_INFO: `/v3/users/988ee615f0104dc4b6e6a4d617093093` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:04.230907 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988ee615f0104dc4b6e6a4d617093093)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:32:04.232887 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:32:04.420377 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988ee615f0104dc4b6e6a4d617093093'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.427915 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988ee615f0104dc4b6e6a4d617093093'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.428650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988ee615f0104dc4b6e6a4d617093093'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.497925 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988ee615f0104dc4b6e6a4d617093093'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.498532 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-55536144-cae2-48de-9fa9-34bca3cc2b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988ee615f0104dc4b6e6a4d617093093'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.516124 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1260/2486] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:32:04 2022] DELETE /identity/v3/users/988ee615f0104dc4b6e6a4d617093093 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:32:04.758403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:32:04.758566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:04.771350 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1542939-d6de-404f-bb8d-be55bbdbb00a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:32:04.827792 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:32:04.828418 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:04.828698 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:04.828975 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] PATH_INFO: `/v3/projects/85c679ebc0954f48a6fd364ee5924df5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:04.830072 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c679ebc0954f48a6fd364ee5924df5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:32:04.832001 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:32:04.931628 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c679ebc0954f48a6fd364ee5924df5', 'request_id': 'req-070052e8-ee58-445e-a685-cfbd003bad41'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.937741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c679ebc0954f48a6fd364ee5924df5', 'request_id': 'req-070052e8-ee58-445e-a685-cfbd003bad41'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:04.944023 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c679ebc0954f48a6fd364ee5924df5', 'request_id': 'req-070052e8-ee58-445e-a685-cfbd003bad41'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:05.034278 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] The token cache is being invalidate because project 85c679ebc0954f48a6fd364ee5924df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:32:05.034951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:05.035237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:32:05.036711 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1227/2487] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:32:04 2022] DELETE /identity/v3/projects/85c679ebc0954f48a6fd364ee5924df5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:32:14.066577 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:14.067104 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:14.067320 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:14.086572 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] MFA Rules not processed for user `18665db956ab4da28230c4c2bfe8ffa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:32:14.087787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:14.130894 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:32:14.141492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1261/2488] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:32:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:32:14.378029 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:32:14.379569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:14.436367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070052e8-ee58-445e-a685-cfbd003bad41 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:32:14.439917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:32:14.440514 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:32:14.440793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:32:14.441064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:32:14.442735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:32:14.452012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:32:14.453423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:32:14.461205 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1228/2489] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:32:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:32:24.258095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:32:24.457670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=80,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:33:18.770714 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:33:18.772464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:33:18.828230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf0ab-cadd-4518-8729-872844a25cc5 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:33:18.831800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:33:18.832391 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:33:18.832664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:33:18.832937 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:33:18.834631 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:33:18.903591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:33:18.905013 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:33:18.915858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1262/2490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:33:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:33:23.046498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:33:23.056279 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68665c1a-5117-4d1b-9171-aed0550c6bee service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:33:23.059878 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:33:23.060463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:33:23.060740 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:33:23.061009 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:33:23.062953 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:33:23.076629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:33:23.078048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:33:23.090014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1229/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:33:24.585858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:33:24.591451 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:33:24.655242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888232e4-3f02-4560-9299-a0543954ec7e service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:33:24.660327 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:33:24.660944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:33:24.661224 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:33:24.661494 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] PATH_INFO: `/v3/users/18665db956ab4da28230c4c2bfe8ffa7` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:33:24.662593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=18665db956ab4da28230c4c2bfe8ffa7)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:33:24.664538 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:33:24.869040 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:33:24.876705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18665db956ab4da28230c4c2bfe8ffa7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:24.878423 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18665db956ab4da28230c4c2bfe8ffa7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:24.879118 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18665db956ab4da28230c4c2bfe8ffa7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:24.982806 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18665db956ab4da28230c4c2bfe8ffa7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:24.983512 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18665db956ab4da28230c4c2bfe8ffa7'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.002429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1263/2492] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:33:24 2022] DELETE /identity/v3/users/18665db956ab4da28230c4c2bfe8ffa7 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:33:25.287613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:33:25.298160 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fd768-260d-4947-aeff-72eaa43b7717 service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:33:25.354169 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:33:25.354798 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:33:25.355104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:33:25.355375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] PATH_INFO: `/v3/projects/044799c63ce94aa39ea143e1633e2002` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:33:25.356475 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044799c63ce94aa39ea143e1633e2002)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:33:25.358356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:33:25.472283 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044799c63ce94aa39ea143e1633e2002', 'request_id': 'req-6d0739ad-5369-4444-9541-0cfdfceb0a77'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.478748 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044799c63ce94aa39ea143e1633e2002', 'request_id': 'req-6d0739ad-5369-4444-9541-0cfdfceb0a77'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.486070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044799c63ce94aa39ea143e1633e2002', 'request_id': 'req-6d0739ad-5369-4444-9541-0cfdfceb0a77'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.590929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] The token cache is being invalidate because project 044799c63ce94aa39ea143e1633e2002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:33:25.591164 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.591821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6d0739ad-5369-4444-9541-0cfdfceb0a77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:33:25.593636 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1230/2493] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:33:25 2022] DELETE /identity/v3/projects/044799c63ce94aa39ea143e1633e2002 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:33:34.998037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=4,keystone:SELECT=13,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:33:35.587880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:34:28.377957 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:28.382400 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:28.458563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d6780-f847-4c22-8683-344577b66efa admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:28.530117 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:28.530745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:28.531056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:28.531333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:28.533213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:28.594731 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:28.596382 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:28.607460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1264/2494] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:33.885537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:33.886091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:33.886203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:33.906340 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] MFA Rules not processed for user `80087d6fbe284b02b20c64ce3237d3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:34:33.907620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:33.914615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1231/2495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:34:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:34:34.210488 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:34.211942 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:34.250663 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=100 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:34:34.266763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ffbff6-0439-44a1-ae21-77f4e2aeb4ef service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:34.270311 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:34.270941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:34.271221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:34.271492 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:34.273668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:34.283896 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:34.285303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:34.294049 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1265/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:34.686909 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:34.688416 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:34.764351 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a71db-3a59-49ef-a161-9da4f4e14ecf None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:34.769374 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:34.769981 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:34.770262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:34.770536 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:34.772338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:34.784323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:34.785904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:34.795811 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1232/2497] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:37.788695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:37.797974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22caee9f-3568-4eb6-8816-040aa9127b7f service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:37.801692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:37.802303 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:37.802737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:37.803054 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:37.805607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:37.819025 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:37.820493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:37.828591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1266/2498] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:44.791566 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=35 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:34:46.783723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:46.785327 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:46.800112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1212e-895a-438c-875a-3c46ae99729b service placement] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:46.805402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:46.805988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:46.806263 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:46.806532 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:46.808270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:46.820518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:46.821986 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:46.832876 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1233/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:47.824786 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=16 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:34:51.018114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:34:51.019992 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:51.036393 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fad95-e607-43a0-967f-5fc19f3bd020 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:34:51.042744 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:34:51.043371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:34:51.043653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:34:51.046701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:34:51.046701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:34:51.058902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:34:51.060473 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:34:51.071375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1267/2500] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:34:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:34:56.828669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=6 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:35:01.067437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=6 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:28.693080 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:28.738008 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdaa8f1-3571-4661-802e-074bbe0688fd service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:28.744453 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03aad2f885bc4dc183e934469676a42f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:28.745086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:28.745366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:28.745643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:28.751734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:28.794375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:28.795955 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:28.811952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1234/2501] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:35.153039 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:35.154476 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:35.214895 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd811e7-2bd5-43eb-a862-ef3863bcc252 service nova] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:35.218443 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:35.219059 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:35.219337 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:35.219603 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:35.221278 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:35.281749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:35.283189 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:35.293034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1268/2502] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:38.803472 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=28 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:38.837118 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:38.838652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:38.884321 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc479a0d-1c9a-4d6b-9edb-7e63b036890d service nova] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:38.888873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:38.889492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:38.889773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:38.890046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] PATH_INFO: `/v3/users/80087d6fbe284b02b20c64ce3237d3b5` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:38.891251 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=80087d6fbe284b02b20c64ce3237d3b5)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:38.893084 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:39.081855 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80087d6fbe284b02b20c64ce3237d3b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.089109 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80087d6fbe284b02b20c64ce3237d3b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.089761 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80087d6fbe284b02b20c64ce3237d3b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.154015 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80087d6fbe284b02b20c64ce3237d3b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.154673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-f6c04342-1283-4165-a7c6-4164eddd5c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80087d6fbe284b02b20c64ce3237d3b5'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.172880 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1235/2503] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:38 2022] DELETE /identity/v3/users/80087d6fbe284b02b20c64ce3237d3b5 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:39.429722 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:39.460827 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f39a-0a89-4041-bb18-63ac06c914e5 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:39.515710 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:39.516362 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:39.516644 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:39.516916 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] PATH_INFO: `/v3/projects/b78491741fc7450b932b7a43e5d3bb92` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:39.518037 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78491741fc7450b932b7a43e5d3bb92)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:39.520251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:39.638375 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:39.650637 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78491741fc7450b932b7a43e5d3bb92', 'request_id': 'req-10e6fc9a-9cf5-48b3-ab78-f598829dd326'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.656766 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78491741fc7450b932b7a43e5d3bb92', 'request_id': 'req-10e6fc9a-9cf5-48b3-ab78-f598829dd326'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.662081 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78491741fc7450b932b7a43e5d3bb92', 'request_id': 'req-10e6fc9a-9cf5-48b3-ab78-f598829dd326'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.741442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] The token cache is being invalidate because project b78491741fc7450b932b7a43e5d3bb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:39.741890 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.743112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-10e6fc9a-9cf5-48b3-ab78-f598829dd326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:39.744612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1269/2504] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:39 2022] DELETE /identity/v3/projects/b78491741fc7450b932b7a43e5d3bb92 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:39.749508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:39.753513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:39.753513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:39.806309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:39.807553 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:39.865490 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1236/2505] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:39.873562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:39.880670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:39.880670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:39.894898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:39.896106 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:39.904064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1270/2506] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:39.908881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:39.918385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4789c-02ec-48ac-bfc5-226b747dd557 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:39.923056 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:39.923883 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:39.924070 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:39.924501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:39.925039 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:36:39.925338 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:39.927430 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:39.936751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:39.939967 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1237/2507] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:36:39.944988 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:39.954462 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add898c3-87ea-4885-a90b-74d69aeda17b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:39.959606 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:39.960204 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:39.960484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:39.960755 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:39.961413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:39.963349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.035945 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1271/2508] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:39 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:40.040493 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.050474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90562da2-d73e-4989-bc3d-4da980bdd4e0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.105818 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.106406 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.106684 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.106987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.107610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.109492 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.216423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1238/2509] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:40 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:36:40.220669 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.231264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec38175-814e-43c1-9c03-f7babf82f264 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.235852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.236442 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.236721 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.236993 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.237626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.239236 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.249460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1272/2510] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:40.253459 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.263100 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca708991-fb6b-4c3b-a637-dd4a2548c1ec admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.267640 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.268224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.268754 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.269026 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.269658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.271271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1761905-be30-4db3-a83d-ee78e4794a57 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.281742 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1239/2511] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:40.285653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.295789 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dd365-6e14-4034-b5e5-70aeb4d17250 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.300246 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.300871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.301149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.301419 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] PATH_INFO: `/v3/projects/0c892355033146ab9351e1790a5bb310/users/d19b1b246f51470883f3b7b1f74674bf/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.316036 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c892355033146ab9351e1790a5bb310, user_id=d19b1b246f51470883f3b7b1f74674bf, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.318683 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.439188 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1273/2512] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:40 2022] PUT /identity/v3/projects/0c892355033146ab9351e1790a5bb310/users/d19b1b246f51470883f3b7b1f74674bf/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:36:40.449589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.449900 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.450172 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.478112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] MFA Rules not processed for user `d19b1b246f51470883f3b7b1f74674bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:40.479358 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:40.540558 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1240/2513] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:40.544928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.604162 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386b0dd-b0f7-421d-867e-c8755ee88210 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.608670 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.609252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.609526 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.609792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.610399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.612328 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.718738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1274/2514] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:40 2022] POST /identity/v3/projects => generated 369 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:36:40.719803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.730933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7142baa5-9aa2-4444-821b-9778511684a2 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.787149 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.787760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.788061 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.788331 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.788963 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.790973 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.896863 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:40.902866 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1241/2515] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:40 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:36:40.906835 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.917181 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283857-6d58-4b26-8219-c0608de662c3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.921680 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.922270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.922548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.922837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.923483 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.925064 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:40.935385 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1275/2516] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:40.939656 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:40.950287 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba072391-33a0-4304-934b-28d73152f10b admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:40.955091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:40.955734 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:40.956011 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:40.956282 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] PATH_INFO: `/v3/projects/ca1eda3c5ec84367ae50084d004c90ce/users/8fd5e70e93524f42ae761b2d494162d6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:40.971744 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1eda3c5ec84367ae50084d004c90ce, user_id=8fd5e70e93524f42ae761b2d494162d6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:40.974393 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:41.056721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1242/2517] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:40 2022] PUT /identity/v3/projects/ca1eda3c5ec84367ae50084d004c90ce/users/8fd5e70e93524f42ae761b2d494162d6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:41.060645 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:41.071029 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9456f327-8c32-4ea9-ba88-597ddb58913d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:41.099338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:41.125213 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:41.125804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:41.126083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:41.126352 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:41.127014 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:41.128594 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8183a53-ff48-4d8d-8e5c-ba5bc0c90330 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:41.139073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1276/2518] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:41.142662 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:41.153970 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658a67b-32ec-4d92-aa9b-1b61832e8d8e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:41.158633 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:41.159295 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:41.159575 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:41.159846 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] PATH_INFO: `/v3/projects/ca1eda3c5ec84367ae50084d004c90ce/users/8fd5e70e93524f42ae761b2d494162d6/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:41.161826 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1eda3c5ec84367ae50084d004c90ce, user_id=8fd5e70e93524f42ae761b2d494162d6, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:41.164657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:41.174749 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1243/2519] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:41 2022] PUT /identity/v3/projects/ca1eda3c5ec84367ae50084d004c90ce/users/8fd5e70e93524f42ae761b2d494162d6/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:41.186837 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:41.187127 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:41.187404 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:41.214557 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] MFA Rules not processed for user `8fd5e70e93524f42ae761b2d494162d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:41.215796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:41.282401 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1277/2520] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:41.293591 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:41.295355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:41.371868 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e255802-b0b1-4be1-b821-57719b5bb01c admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:41.375479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:41.376067 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:41.376341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:41.376608 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:41.385879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:41.387334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250de1-70e9-46ec-9a0d-edbcc7e6c1ea service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:41.396795 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1244/2521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:42.284466 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:42.345251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d21ce9-af7c-4b9e-beac-ae17c7308f85 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:42.349758 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:42.350376 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:42.350659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:42.350968 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] PATH_INFO: `/v3/users/d19b1b246f51470883f3b7b1f74674bf` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:42.352083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19b1b246f51470883f3b7b1f74674bf)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:42.353981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:42.533748 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19b1b246f51470883f3b7b1f74674bf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:42.539098 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:42.541028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19b1b246f51470883f3b7b1f74674bf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:42.541720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19b1b246f51470883f3b7b1f74674bf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:42.627703 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19b1b246f51470883f3b7b1f74674bf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:42.628394 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19b1b246f51470883f3b7b1f74674bf'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:42.646844 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1278/2522] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:42 2022] DELETE /identity/v3/users/d19b1b246f51470883f3b7b1f74674bf => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:42.651612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:42.652077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:42.652181 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:42.671793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:42.672989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:42.732392 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1245/2523] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:42.745460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:42.746877 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:42.804085 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2c0ea-a32c-4239-980e-045d9dce99b9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:42.807650 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:42.808249 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:42.808525 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:42.808796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:42.818270 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:42.819725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:42.829148 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1279/2524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:43.229869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:43.231308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:43.245724 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886713a-9c05-4f73-bfe0-c9dfe0f95a93 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:43.251228 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:43.251997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:43.252174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:43.252452 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] PATH_INFO: `/v3/projects/0c892355033146ab9351e1790a5bb310` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:43.253610 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c892355033146ab9351e1790a5bb310)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:43.255741 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:43.358664 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:43.365949 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c892355033146ab9351e1790a5bb310', 'request_id': 'req-2e6381ba-f647-4696-9641-6f8d75f50cf3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.372207 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c892355033146ab9351e1790a5bb310', 'request_id': 'req-2e6381ba-f647-4696-9641-6f8d75f50cf3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.377696 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c892355033146ab9351e1790a5bb310', 'request_id': 'req-2e6381ba-f647-4696-9641-6f8d75f50cf3'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.443859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] The token cache is being invalidate because project 0c892355033146ab9351e1790a5bb310 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:43.444143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.444759 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.446225 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1246/2525] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:43 2022] DELETE /identity/v3/projects/0c892355033146ab9351e1790a5bb310 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:43.450145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:43.460661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca61978-191b-40cd-a7a5-fbe2d94b6208 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:43.515974 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:43.516601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:43.516878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:43.517153 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] PATH_INFO: `/v3/users/8fd5e70e93524f42ae761b2d494162d6` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:43.518507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd5e70e93524f42ae761b2d494162d6)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:43.520966 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:43.716277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd5e70e93524f42ae761b2d494162d6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.723034 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd5e70e93524f42ae761b2d494162d6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.724304 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd5e70e93524f42ae761b2d494162d6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.782454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd5e70e93524f42ae761b2d494162d6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.783120 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-03efbdf2-5ea6-4196-a2a3-bf34320253bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd5e70e93524f42ae761b2d494162d6'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:43.793852 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:43.801353 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1280/2526] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:43 2022] DELETE /identity/v3/users/8fd5e70e93524f42ae761b2d494162d6 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:44.177844 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.179339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:44.193130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6381ba-f647-4696-9641-6f8d75f50cf3 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.248877 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.249437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.249743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.250014 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] PATH_INFO: `/v3/projects/ca1eda3c5ec84367ae50084d004c90ce` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.251174 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1eda3c5ec84367ae50084d004c90ce)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.253065 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.329927 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1eda3c5ec84367ae50084d004c90ce', 'request_id': 'req-77b97b7e-227b-468c-bd00-c850abee8c40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:44.336000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1eda3c5ec84367ae50084d004c90ce', 'request_id': 'req-77b97b7e-227b-468c-bd00-c850abee8c40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:44.341402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1eda3c5ec84367ae50084d004c90ce', 'request_id': 'req-77b97b7e-227b-468c-bd00-c850abee8c40'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:44.398219 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] The token cache is being invalidate because project ca1eda3c5ec84367ae50084d004c90ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:44.398501 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:44.399137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-77b97b7e-227b-468c-bd00-c850abee8c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:44.400594 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1247/2527] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:44 2022] DELETE /identity/v3/projects/ca1eda3c5ec84367ae50084d004c90ce => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:44.405548 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.405841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.407033 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.427485 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:44.428720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:44.485501 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1281/2528] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:44.493642 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.493928 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.494201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.514020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:44.515192 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:44.523233 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1248/2529] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:44.527244 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.537218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2369-f696-4623-97e3-1be9a1fad28b None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.541897 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.542484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.542760 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.543066 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.543697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:36:44.543997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.545595 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.557439 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1282/2530] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:36:44.561491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.570497 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd63126-c0e7-4153-89d9-05f6bdb449bd None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.575057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.575646 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.575924 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.576196 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.576824 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.578695 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.669215 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1249/2531] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:44 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:44.673281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.683633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1498-3e16-43ba-bba1-02e5e820e24b admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.737728 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.738354 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.738630 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.739612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.740230 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.742123 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.816194 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1283/2532] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:44 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:44.820203 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.830579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836853f-f7bb-458d-8afa-8564697c34fd admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.835200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.835789 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.836068 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.836339 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.836968 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.838551 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.848803 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1250/2533] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:44.852930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.862563 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0e05c-f8d7-4e11-baa3-9146bcf16806 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.867900 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.868509 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.868791 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.869074 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.869705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.871324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92a026-e546-4c70-978f-1a488cda209f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.882223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1284/2534] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:44.886434 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:44.897048 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d29b0-7dfe-4cd4-891e-e4a64b6ef74a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:44.901654 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:44.902289 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:44.902568 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:44.902865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] PATH_INFO: `/v3/projects/fb198d42e0c74cfba2b91e58b524cad3/users/852b2df67dc0404792ad679fc254d16b/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:44.917616 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb198d42e0c74cfba2b91e58b524cad3, user_id=852b2df67dc0404792ad679fc254d16b, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:44.920348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:44.922538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:44.998895 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1251/2535] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:44 2022] PUT /identity/v3/projects/fb198d42e0c74cfba2b91e58b524cad3/users/852b2df67dc0404792ad679fc254d16b/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:45.009091 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.009374 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.009682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.038176 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] MFA Rules not processed for user `852b2df67dc0404792ad679fc254d16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:45.039417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:45.084729 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:45.102260 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1285/2536] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:45.106843 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.167600 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c55d8-f7ae-4fdb-abd1-dc5c33eb9c58 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.172545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.173139 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.173419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.173692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.174305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.176214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.246341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1252/2537] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:45 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:45.250175 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.261203 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92c825-80e5-4c8a-8705-2e33b3b752cc None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.315625 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.316221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.316503 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.316778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.317931 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.319879 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.416038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1286/2538] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:45 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:45.420081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.430443 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2b8d5-16ae-404b-a0d2-2be84ecb8416 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.435281 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.435875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.436154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.436426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.437057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.438638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.449599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1253/2539] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:45.453709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.464095 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5ad03-b972-4707-9fad-cf3108e0757c admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.468653 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.469282 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.469562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.469829 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] PATH_INFO: `/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3/users/c1f88cccd57c4cab850031176d5f25a8/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.484146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efcd47c49b74fd4b9aaf625e31b50a3, user_id=c1f88cccd57c4cab850031176d5f25a8, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.486749 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.564309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1287/2540] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:45 2022] PUT /identity/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3/users/c1f88cccd57c4cab850031176d5f25a8/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:45.568990 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.578912 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf4cb-ef0b-481f-ab7d-f34a4a4364e6 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.633461 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.634057 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.634333 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.634603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.635270 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.636827 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c68d4-79d3-467d-9248-6966a3e1327a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.647107 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1254/2541] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:45.651338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.661535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d5c3f-222e-4b87-930e-e9299ee7ebd8 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.666271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.666860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.667142 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.667420 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] PATH_INFO: `/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3/users/c1f88cccd57c4cab850031176d5f25a8/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.669386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0efcd47c49b74fd4b9aaf625e31b50a3, user_id=c1f88cccd57c4cab850031176d5f25a8, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.672045 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.682586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1288/2542] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:45 2022] PUT /identity/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3/users/c1f88cccd57c4cab850031176d5f25a8/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:36:45.694094 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.694378 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.694653 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.724388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] MFA Rules not processed for user `c1f88cccd57c4cab850031176d5f25a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:45.724388 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:45.775833 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:45.793987 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1255/2543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:45.805902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:45.807620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:45.863701 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd221792-d4b2-465c-8aa6-e63c860b3802 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:45.868146 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:45.868739 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:45.869019 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:45.869291 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:45.878110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:45.880112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:45.889437 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1289/2544] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:46.046081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:46.056437 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159109c8-75e7-4247-8dbf-5386640e95e0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:46.060060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:46.060657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:46.060961 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:46.061234 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:46.116441 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:46.117861 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ef3df-6cb4-4d2f-a58a-442cb5c30ca9 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:46.126579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1256/2545] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:49.554633 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:49.598695 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:49.614345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c1e5e-eb6d-4aa8-945c-0695b0adc6c5 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:49.618795 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:49.622392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:49.622392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:49.622392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] PATH_INFO: `/v3/users/852b2df67dc0404792ad679fc254d16b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:49.622392 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=852b2df67dc0404792ad679fc254d16b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:49.622938 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:49.834958 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852b2df67dc0404792ad679fc254d16b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:49.841655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852b2df67dc0404792ad679fc254d16b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:49.842334 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852b2df67dc0404792ad679fc254d16b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:49.924035 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852b2df67dc0404792ad679fc254d16b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:49.924705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852b2df67dc0404792ad679fc254d16b'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:49.943147 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1290/2546] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:49 2022] DELETE /identity/v3/users/852b2df67dc0404792ad679fc254d16b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:49.947701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:49.948436 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:49.948613 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:49.968091 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:49.969277 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:50.026272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1257/2547] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:50.035847 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:50.037223 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:50.093987 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9bea7-55fb-4343-bfd2-508778df1454 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:50.097418 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:50.097997 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:50.098271 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:50.098537 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:50.107371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:50.108770 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:50.117917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1291/2548] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:50.468310 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:50.469728 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:50.484042 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452519bc-7886-414a-b156-ed698edf4a87 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:50.489692 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:50.490508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:50.490884 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:50.491086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] PATH_INFO: `/v3/projects/fb198d42e0c74cfba2b91e58b524cad3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:50.492280 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb198d42e0c74cfba2b91e58b524cad3)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:50.494218 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:50.592720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb198d42e0c74cfba2b91e58b524cad3', 'request_id': 'req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.599614 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb198d42e0c74cfba2b91e58b524cad3', 'request_id': 'req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.605165 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb198d42e0c74cfba2b91e58b524cad3', 'request_id': 'req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.689025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] The token cache is being invalidate because project fb198d42e0c74cfba2b91e58b524cad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:50.689309 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.689921 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.691962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1258/2549] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:50 2022] DELETE /identity/v3/projects/fb198d42e0c74cfba2b91e58b524cad3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:50.696110 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:50.705893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473f6-f392-42a8-90d0-513d224fd595 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:50.762015 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:50.762641 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:50.763028 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:50.763306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] PATH_INFO: `/v3/users/c1f88cccd57c4cab850031176d5f25a8` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:50.764405 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f88cccd57c4cab850031176d5f25a8)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:50.766306 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:50.771771 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:50.988843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f88cccd57c4cab850031176d5f25a8'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:50.996489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f88cccd57c4cab850031176d5f25a8'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.008053 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f88cccd57c4cab850031176d5f25a8'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.061389 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f88cccd57c4cab850031176d5f25a8'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.062010 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7d7f7725-a056-47a6-a2e5-af8dc2cb902e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f88cccd57c4cab850031176d5f25a8'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.079866 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1292/2550] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:50 2022] DELETE /identity/v3/users/c1f88cccd57c4cab850031176d5f25a8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:51.434456 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:51.436155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:51.449874 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb3eed-89b8-444c-8c69-3043ec43c3fe admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:51.458933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:51.508252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:51.508857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:51.509141 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:51.509444 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] PATH_INFO: `/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:51.510577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0efcd47c49b74fd4b9aaf625e31b50a3)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:51.512522 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:51.643419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0efcd47c49b74fd4b9aaf625e31b50a3', 'request_id': 'req-107bcbf4-a228-4e05-8f9d-7a42387582ff'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.650064 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0efcd47c49b74fd4b9aaf625e31b50a3', 'request_id': 'req-107bcbf4-a228-4e05-8f9d-7a42387582ff'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.655561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0efcd47c49b74fd4b9aaf625e31b50a3', 'request_id': 'req-107bcbf4-a228-4e05-8f9d-7a42387582ff'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.734948 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] The token cache is being invalidate because project 0efcd47c49b74fd4b9aaf625e31b50a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:51.735238 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.735853 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-107bcbf4-a228-4e05-8f9d-7a42387582ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:51.737348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1259/2551] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:51 2022] DELETE /identity/v3/projects/0efcd47c49b74fd4b9aaf625e31b50a3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:51.742941 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:51.743413 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:51.743528 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:51.764253 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:51.765386 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:51.824113 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1293/2552] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:51.832252 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:51.832545 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:51.832826 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:51.852607 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:51.853815 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:51.861947 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1260/2553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:51.866785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:51.876345 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bc414-9bdf-4a44-9270-3091ecbc772e None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:51.881562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:51.882157 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:51.882436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:51.882709 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:51.883472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:36:51.883783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:51.885372 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:51.897559 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1294/2554] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:36:51.902050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:51.911449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0e9c9-2276-4305-ae44-fe6a3f4b83d8 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:51.916145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:51.916739 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:51.917021 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:51.917341 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:51.917976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:51.920133 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.011585 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1261/2555] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:51 2022] POST /identity/v3/projects => generated 361 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:52.016052 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.026447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b15f1-bca3-4c82-861f-0993030b0d02 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.081596 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.082192 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.082499 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.082772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.083433 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.085338 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.164717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1295/2556] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:52 2022] POST /identity/v3/users => generated 281 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:36:52.168994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.180046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a7a7f-453e-4742-9fae-3a253373d567 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.184606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.185214 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.185491 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.185760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.186389 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.188000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.197735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1262/2557] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:52.201573 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.206745 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:52.212173 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dea193-ca38-4c96-9ebd-5f6f9a2baaae admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.216845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.217429 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.221378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.221378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.221378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.221378 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d0811-a342-4bc7-b602-c9dc491cabcc admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.231520 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1296/2558] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:52.235200 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.245703 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ed7bf-95c3-4777-992a-c4cc8197cf0e admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.250294 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.250956 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.251239 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.251516 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] PATH_INFO: `/v3/projects/603d4f48ec6f4aae9dd9ed2335c46fb9/users/8cc8b2a6250f461aa48245d80a2884ec/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.265697 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=603d4f48ec6f4aae9dd9ed2335c46fb9, user_id=8cc8b2a6250f461aa48245d80a2884ec, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.268638 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.318028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1263/2559] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:52 2022] PUT /identity/v3/projects/603d4f48ec6f4aae9dd9ed2335c46fb9/users/8cc8b2a6250f461aa48245d80a2884ec/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:52.381438 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.381706 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.381977 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.411154 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] MFA Rules not processed for user `8cc8b2a6250f461aa48245d80a2884ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:52.412381 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:52.474261 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1297/2560] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 26 11:36:52.507099 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.508892 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:52.550479 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:52.565000 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c632db3-ffde-47ef-b142-99d54be21abb admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.568719 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.569317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.569599 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.569872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.577936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.579449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3203c76-d266-4e11-a608-fa1caf87a896 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.588788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1264/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:52.604935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:52.665924 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7fbd8-0f52-4756-a83b-7959cefcdddd None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:52.670482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:52.671135 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.671417 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.671691 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] PATH_INFO: `/v3/users/8cc8b2a6250f461aa48245d80a2884ec` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.672808 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc8b2a6250f461aa48245d80a2884ec)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:52.674713 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:52.869323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc8b2a6250f461aa48245d80a2884ec'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:52.876294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc8b2a6250f461aa48245d80a2884ec'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:52.876961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc8b2a6250f461aa48245d80a2884ec'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:52.952723 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc8b2a6250f461aa48245d80a2884ec'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:52.953349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc8b2a6250f461aa48245d80a2884ec'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:52.967763 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:52.971923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1298/2562] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:36:52 2022] DELETE /identity/v3/users/8cc8b2a6250f461aa48245d80a2884ec => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:52.976348 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:52.979554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:52.979554 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:52.995379 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:52.996569 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:53.055773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1265/2563] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:53.065399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:53.067324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:53.123881 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c3861-5d13-4a53-9d45-1e5909f34532 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:53.127935 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:53.128562 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:53.129330 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:53.129599 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:53.138416 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:53.139863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6789eb4-573c-4bbc-b1ff-0fd506d524fe service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:53.149981 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1299/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:36:53.571450 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:53.572896 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:53.586425 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda029cc-c6e0-45fc-b109-a8941542b687 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:53.591780 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:53.592398 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:53.592674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:53.592941 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] PATH_INFO: `/v3/projects/603d4f48ec6f4aae9dd9ed2335c46fb9` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:53.594088 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=603d4f48ec6f4aae9dd9ed2335c46fb9)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:53.596006 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:53.689541 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603d4f48ec6f4aae9dd9ed2335c46fb9', 'request_id': 'req-3a6d9084-fb4f-454c-b808-5fd846c47bdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:53.695609 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603d4f48ec6f4aae9dd9ed2335c46fb9', 'request_id': 'req-3a6d9084-fb4f-454c-b808-5fd846c47bdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:53.701063 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603d4f48ec6f4aae9dd9ed2335c46fb9', 'request_id': 'req-3a6d9084-fb4f-454c-b808-5fd846c47bdf'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:53.821917 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] The token cache is being invalidate because project 603d4f48ec6f4aae9dd9ed2335c46fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:36:53.822209 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:53.822839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-3a6d9084-fb4f-454c-b808-5fd846c47bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:36:53.824375 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1266/2565] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:36:53 2022] DELETE /identity/v3/projects/603d4f48ec6f4aae9dd9ed2335c46fb9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:53.829269 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:53.829720 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:53.829999 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:53.851730 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:53.852936 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:53.911126 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1300/2566] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:53.918686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:53.918997 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:53.919272 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:53.939130 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:53.940376 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:53.948117 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1267/2567] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:53.952216 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:53.962428 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a66d0-7d76-4b85-abb2-f951385a006f None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:53.966885 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:53.967472 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:53.967747 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:53.968020 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:53.968654 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:36:53.968951 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:53.970549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:53.982350 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1301/2568] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:36:53.986363 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:53.995439 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b06e1-c8ab-44b3-9ef6-11b2d6edabd0 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.000137 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.000718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.000992 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.001261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.001873 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.003775 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.078405 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1268/2569] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:53 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:54.083144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.092817 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fac26-3999-4e4b-83f1-62435295955f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.147491 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.148080 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.148355 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.148624 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.149240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.151158 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.220875 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1302/2570] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:54 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:36:54.225705 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.235885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e6525-4fc3-4d34-bf5a-5059697fe26a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.240391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.240974 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.241249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.241518 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.242143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.243869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.254012 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1269/2571] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:54.257787 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.267928 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a664e73-4f95-496a-8974-543853219b68 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.273317 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.273955 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.274206 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.274477 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.275141 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.276712 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a9b8-3008-401a-8e99-1ac5affdbd3a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.287149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1303/2572] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:54.291201 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.300988 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560ad8d-5084-4526-80d6-ec3422ebb0ba admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.305505 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.306132 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.306410 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.306679 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] PATH_INFO: `/v3/projects/04d1cfa593d0460dac4e3548e98ab03f/users/6f4e2cc1dfa9409bb3285cafe83f993a/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.321129 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d1cfa593d0460dac4e3548e98ab03f, user_id=6f4e2cc1dfa9409bb3285cafe83f993a, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.323773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.525989 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:54.530590 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1270/2573] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:54 2022] PUT /identity/v3/projects/04d1cfa593d0460dac4e3548e98ab03f/users/6f4e2cc1dfa9409bb3285cafe83f993a/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:36:54.547697 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.547975 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.548242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.567725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:54.576336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] MFA Rules not processed for user `6f4e2cc1dfa9409bb3285cafe83f993a`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:54.583648 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:54.640969 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1304/2574] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:54.645729 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.705652 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb3fe1-b5be-408a-9d34-43ab322aa1e0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.710152 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.710743 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.711046 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.711323 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.711932 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.713801 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.839735 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1271/2575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:54.843634 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:54.854456 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb50045-8ce9-42b0-a096-ecdeaea92236 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:54.908613 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:54.909209 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:54.909484 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:54.909750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:54.910371 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:54.912323 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:54.992705 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1305/2576] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:54 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:54.996760 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.007706 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8680cf-1fc1-421d-9578-c9737a9f794a admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.012284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.012872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.013151 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.013423 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.014102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.015718 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.026305 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1272/2577] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:55.030072 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.040290 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf59d0a-1f58-4849-a5b0-434afc740a7f admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.045275 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.045858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.046131 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.046397 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.047051 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.048626 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49efa6f-6bea-434f-b350-86a531889b69 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.059083 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1306/2578] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:55.063025 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.072839 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806fa7-e4ae-407b-89c8-e978145476f1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.077442 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.078082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.078367 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.078647 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] PATH_INFO: `/v3/projects/d953bcda51a94cf19476b0b60b91912e/users/f3436807d774406b9acd63a98e8988f7/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.092886 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d953bcda51a94cf19476b0b60b91912e, user_id=f3436807d774406b9acd63a98e8988f7, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.095502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.181322 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1273/2579] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:55 2022] PUT /identity/v3/projects/d953bcda51a94cf19476b0b60b91912e/users/f3436807d774406b9acd63a98e8988f7/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:55.192692 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.192979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.193251 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.220961 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] MFA Rules not processed for user `f3436807d774406b9acd63a98e8988f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:55.222247 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:55.284607 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1307/2580] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:55.289589 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.349509 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b927dbc-3e7d-42d8-9988-c93848f2ab11 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.354020 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.354611 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.354920 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.355190 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.355800 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.357686 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.450355 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1274/2581] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:36:55 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:55.454373 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.465467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bf998-7cfc-47b3-ae8a-829610f4daed None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.470619 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:55.520783 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.523351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.523351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.523351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.523351 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.524489 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.626602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1308/2582] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:36:55 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:36:55.630606 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.640944 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b2628-5377-47fb-84b6-5e315ef74201 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.646150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.646745 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.647036 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.647307 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.647929 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.649502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.659770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1275/2583] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:55.664498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.674227 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d50c04-0bd5-4da5-9d4b-6edbda83d81d admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.678655 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.679316 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.679591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.679860 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] PATH_INFO: `/v3/projects/3f6c904b93624cc7831d936f43ef6d0b/users/d2f2050e57c44633bbce91eb16d82a82/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.694149 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6c904b93624cc7831d936f43ef6d0b, user_id=d2f2050e57c44633bbce91eb16d82a82, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.696800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.752820 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1309/2584] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:55 2022] PUT /identity/v3/projects/3f6c904b93624cc7831d936f43ef6d0b/users/d2f2050e57c44633bbce91eb16d82a82/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:55.757356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.767538 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850958b3-5db1-4739-933d-b5f5a1781f09 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.787284 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:55.822529 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.823154 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.823431 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.823717 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.824325 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.825897 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738b172-6f90-4873-be35-5a8a381c95da admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.837387 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1276/2585] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:36:55.841112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.851163 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e9a9d-5641-4f77-82bf-e07e9b4b5217 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:55.855930 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:55.856567 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.856843 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.857112 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] PATH_INFO: `/v3/projects/3f6c904b93624cc7831d936f43ef6d0b/users/d2f2050e57c44633bbce91eb16d82a82/roles/27a106926ca14908b7e268013b8452a2` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.859026 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6c904b93624cc7831d936f43ef6d0b, user_id=d2f2050e57c44633bbce91eb16d82a82, role_id=27a106926ca14908b7e268013b8452a2)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:55.861668 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:55.871682 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1310/2586] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:36:55 2022] PUT /identity/v3/projects/3f6c904b93624cc7831d936f43ef6d0b/users/d2f2050e57c44633bbce91eb16d82a82/roles/27a106926ca14908b7e268013b8452a2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:36:55.882182 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:55.882409 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:55.882682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:55.910262 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] MFA Rules not processed for user `d2f2050e57c44633bbce91eb16d82a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:36:55.911502 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:55.977515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1277/2587] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:36:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:36:55.989460 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:55.997765 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:56.049240 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff79a0a-efe2-4d1b-a4cf-99881eb13bb4 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:56.052796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:56.053399 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:56.053659 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:56.053926 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:56.107368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:56.108782 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:56.117878 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1311/2588] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:58.022890 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:58.024299 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:36:58.036584 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede9d88-e042-44f0-9fe7-40dcc3354314 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:58.040086 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:58.040682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:58.040966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:58.041249 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:58.095371 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:58.096790 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:58.105945 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1278/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:36:59.500612 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:36:59.509564 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795a4fe-040b-4ae0-b177-56bd27f7cf09 service cinder] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:36:59.513214 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:36:59.513802 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:36:59.514075 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:36:59.514340 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:36:59.524032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:36:59.525425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092b4b-1c94-48df-8194-09454d8f8a83 service cinder] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:36:59.530294 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:36:59.534976 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1312/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:36:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:37:04.211204 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:04.255494 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=100 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:04.273035 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c021-50ce-476a-8655-30c9c6f96815 service cinder] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:04.277673 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:04.278293 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:04.278574 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:04.278872 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] PATH_INFO: `/v3/users/6f4e2cc1dfa9409bb3285cafe83f993a` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:04.279976 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4e2cc1dfa9409bb3285cafe83f993a)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:04.281940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:04.439112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4e2cc1dfa9409bb3285cafe83f993a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:04.445851 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4e2cc1dfa9409bb3285cafe83f993a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:04.446495 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4e2cc1dfa9409bb3285cafe83f993a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:04.517783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4e2cc1dfa9409bb3285cafe83f993a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:04.518397 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4e2cc1dfa9409bb3285cafe83f993a'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:04.536904 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1279/2591] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:37:04 2022] DELETE /identity/v3/users/6f4e2cc1dfa9409bb3285cafe83f993a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:04.541368 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:04.541873 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:04.542003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:04.560406 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:04.561592 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:04.618446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1313/2592] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:04.628155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:04.629643 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:04.686354 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78b805-dfbb-40a0-beb7-5ec7bddbd6a1 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:04.689971 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:04.690508 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:04.690783 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:04.691085 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:04.699788 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:04.701189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:04.711271 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1280/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:37:05.072821 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:05.074313 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:05.087200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af258bcb-73d5-4224-8b99-e1341aba6f9c None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:05.091919 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:05.092550 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:05.092856 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:05.093129 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] PATH_INFO: `/v3/projects/04d1cfa593d0460dac4e3548e98ab03f` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:05.094277 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d1cfa593d0460dac4e3548e98ab03f)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:05.096238 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:05.202929 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d1cfa593d0460dac4e3548e98ab03f', 'request_id': 'req-f9836ac0-caed-4884-ae77-abd1508df9e5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.209568 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d1cfa593d0460dac4e3548e98ab03f', 'request_id': 'req-f9836ac0-caed-4884-ae77-abd1508df9e5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.215073 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d1cfa593d0460dac4e3548e98ab03f', 'request_id': 'req-f9836ac0-caed-4884-ae77-abd1508df9e5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.287464 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] The token cache is being invalidate because project 04d1cfa593d0460dac4e3548e98ab03f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:37:05.287750 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.288366 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.289861 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1314/2594] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:37:05 2022] DELETE /identity/v3/projects/04d1cfa593d0460dac4e3548e98ab03f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:05.293807 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:05.303189 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cad59d-4690-4a16-9124-e808f19c551f service neutron] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:05.358463 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:05.359104 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:05.359386 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:05.359657 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] PATH_INFO: `/v3/users/f3436807d774406b9acd63a98e8988f7` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:05.360751 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3436807d774406b9acd63a98e8988f7)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:05.362666 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:05.367891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:05.536419 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3436807d774406b9acd63a98e8988f7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.543353 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3436807d774406b9acd63a98e8988f7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.544022 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3436807d774406b9acd63a98e8988f7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.614577 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3436807d774406b9acd63a98e8988f7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.615317 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3436807d774406b9acd63a98e8988f7'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:05.633050 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1281/2595] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:37:05 2022] DELETE /identity/v3/users/f3436807d774406b9acd63a98e8988f7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:06.067218 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:06.069738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:06.087178 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9836ac0-caed-4884-ae77-abd1508df9e5 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:06.144845 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:06.145510 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:06.145794 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:06.146071 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] PATH_INFO: `/v3/projects/d953bcda51a94cf19476b0b60b91912e` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:06.147212 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d953bcda51a94cf19476b0b60b91912e)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:06.149114 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:06.272746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:06.288513 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd953bcda51a94cf19476b0b60b91912e', 'request_id': 'req-876f2406-b390-464f-8328-62ab81f273d3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.295000 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd953bcda51a94cf19476b0b60b91912e', 'request_id': 'req-876f2406-b390-464f-8328-62ab81f273d3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.300497 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd953bcda51a94cf19476b0b60b91912e', 'request_id': 'req-876f2406-b390-464f-8328-62ab81f273d3'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.372664 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] The token cache is being invalidate because project d953bcda51a94cf19476b0b60b91912e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:37:06.372952 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.373591 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.375089 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1315/2596] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:37:06 2022] DELETE /identity/v3/projects/d953bcda51a94cf19476b0b60b91912e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:06.379102 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:06.381024 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:06.393822 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85b64b-1c47-4fdb-ac7b-6a0b273699aa admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:06.449403 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:06.450040 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:06.450297 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:06.450567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] PATH_INFO: `/v3/users/d2f2050e57c44633bbce91eb16d82a82` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:06.451685 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f2050e57c44633bbce91eb16d82a82)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:06.453579 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:06.673797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f2050e57c44633bbce91eb16d82a82'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.681156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f2050e57c44633bbce91eb16d82a82'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.681805 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f2050e57c44633bbce91eb16d82a82'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.745802 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f2050e57c44633bbce91eb16d82a82'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.746762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-a0952bf2-9dd0-4a70-bff9-e3922124a082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f2050e57c44633bbce91eb16d82a82'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:06.764701 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1282/2597] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:37:06 2022] DELETE /identity/v3/users/d2f2050e57c44633bbce91eb16d82a82 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:07.187257 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.188746 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:07.206407 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2406-b390-464f-8328-62ab81f273d3 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.262912 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.263551 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.263831 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.264107 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] PATH_INFO: `/v3/projects/3f6c904b93624cc7831d936f43ef6d0b` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.265321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6c904b93624cc7831d936f43ef6d0b)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.267533 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.415084 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6c904b93624cc7831d936f43ef6d0b', 'request_id': 'req-7a409603-9739-4690-8443-0b0c2decb981'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:07.421324 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6c904b93624cc7831d936f43ef6d0b', 'request_id': 'req-7a409603-9739-4690-8443-0b0c2decb981'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:07.426863 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6c904b93624cc7831d936f43ef6d0b', 'request_id': 'req-7a409603-9739-4690-8443-0b0c2decb981'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:07.505032 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] The token cache is being invalidate because project 3f6c904b93624cc7831d936f43ef6d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81745) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:37:07.505326 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:07.509772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-7a409603-9739-4690-8443-0b0c2decb981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:07.509772 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1316/2598] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:37:07 2022] DELETE /identity/v3/projects/3f6c904b93624cc7831d936f43ef6d0b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:07.513464 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.513919 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.514028 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.523512 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:07.534232 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:07.535460 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:07.592381 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1283/2599] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:07.601858 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.602144 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.605796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.622021 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:07.623281 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:07.631186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1317/2600] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:07.635620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.645474 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cafbb-91ed-4bbe-a444-88830c5bcf93 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.651298 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.651891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.652166 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.652435 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.653206 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81744) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:37:07.653561 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.655170 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.667062 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1284/2601] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 26 11:37:07.671425 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.680549 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f6213-d5e2-464e-aa1b-724857c74401 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.685191 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.685778 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.686057 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.686333 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.687056 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.689003 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.787684 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1318/2602] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:37:07 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:37:07.791859 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.801875 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89f1ab-4740-4cd1-9271-6bf33b5cf8e0 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.857869 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.858499 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.858781 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.859087 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] PATH_INFO: `/v3/users` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.859723 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.861629 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.928391 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1285/2603] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:37:07 2022] POST /identity/v3/users => generated 285 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:37:07.932482 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.943586 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca1f71-e386-49d5-b7d7-2e3330b2cb19 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.948349 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.948979 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.949259 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.949529 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.950221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.951836 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.962211 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1319/2604] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:37:07.966143 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:07.976049 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dcafb-5f32-4d86-9f9f-c6a52d940836 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:07.980594 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:07.981179 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:07.981454 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:07.981721 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:07.982344 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:07.983940 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a7570-dcf8-4f0b-9453-9f5ef72c2e80 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:07.995511 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1286/2605] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:37:07.999474 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:08.006099 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:08.009717 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0ae41-6a3f-48da-b5ff-3685f755be70 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:08.014390 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:08.015737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] REQUEST_METHOD: `PUT` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:08.015737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:08.015737 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] PATH_INFO: `/v3/projects/1e37cda278c24710a39cd0786c79a269/users/c349fffa2ff544a7a239f5e4c830a3a4/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:08.030593 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e37cda278c24710a39cd0786c79a269, user_id=c349fffa2ff544a7a239f5e4c830a3a4, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:08.033265 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:08.093804 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1320/2606] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:37:07 2022] PUT /identity/v3/projects/1e37cda278c24710a39cd0786c79a269/users/c349fffa2ff544a7a239f5e4c830a3a4/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:08.105682 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:08.105966 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:08.106240 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:08.133224 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] MFA Rules not processed for user `c349fffa2ff544a7a239f5e4c830a3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:08.134429 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:08.193693 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:08.197213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1287/2607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:08.214356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:08.214356 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:08.290038 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b8fdc-bc79-470d-9d29-20fed772a36a admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:08.293602 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59ac752182c14071999766921f6b4f09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:08.294200 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:08.294478 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:08.294753 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:08.302773 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:08.304221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:08.312833 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1321/2608] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:37:18.308876 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:22.722936 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:22.724346 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:22.814857 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081938ac-d5d6-40cd-9b8e-3129134b1faa None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:22.819994 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:22.820603 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:22.820881 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:22.821156 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:22.822933 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:22.838670 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:22.840957 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2358bae-4577-4bfa-90a6-00c075099538 service placement] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:22.850870 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1288/2609] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:37:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:37:32.846604 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=41 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:42.650681 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:42.712279 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bea6a5-082c-4919-a6fa-601dc866ca0a service glance] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:42.717242 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:42.717859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:42.718139 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:42.718410 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] PATH_INFO: `/v3/users/c349fffa2ff544a7a239f5e4c830a3a4` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:42.719543 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c349fffa2ff544a7a239f5e4c830a3a4)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:42.721475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:42.871481 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c349fffa2ff544a7a239f5e4c830a3a4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:42.878515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c349fffa2ff544a7a239f5e4c830a3a4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:42.879215 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c349fffa2ff544a7a239f5e4c830a3a4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:42.937100 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c349fffa2ff544a7a239f5e4c830a3a4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:42.937661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c349fffa2ff544a7a239f5e4c830a3a4'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:42.955221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1322/2610] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:37:42 2022] DELETE /identity/v3/users/c349fffa2ff544a7a239f5e4c830a3a4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:42.959835 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:42.960308 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:42.960426 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:42.979144 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:42.980847 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:43.037567 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1289/2611] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:43.047686 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:43.049055 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:43.105367 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101954d-13f2-4a2c-b6c7-5542ce8be494 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:43.109186 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:43.109796 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.110070 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.110339 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.119601 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:43.122221 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69805103-a0b4-41dc-979b-ae9617299c46 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:43.131793 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1323/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:37:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:37:43.487658 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:43.489081 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:43.501193 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a532c67-7d9d-449c-bc71-24d17d94e127 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:43.505891 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:43.506537 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.506864 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.507145 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] PATH_INFO: `/v3/projects/1e37cda278c24710a39cd0786c79a269` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.508328 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e37cda278c24710a39cd0786c79a269)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:43.510261 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:43.617268 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e37cda278c24710a39cd0786c79a269', 'request_id': 'req-6aabf6ab-772e-4e22-9eff-d126e6449b19'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:43.623520 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e37cda278c24710a39cd0786c79a269', 'request_id': 'req-6aabf6ab-772e-4e22-9eff-d126e6449b19'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:43.629077 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e37cda278c24710a39cd0786c79a269', 'request_id': 'req-6aabf6ab-772e-4e22-9eff-d126e6449b19'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:43.679330 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] The token cache is being invalidate because project 1e37cda278c24710a39cd0786c79a269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:37:43.679620 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:43.680260 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-6aabf6ab-772e-4e22-9eff-d126e6449b19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:37:43.681753 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1290/2613] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:37:43 2022] DELETE /identity/v3/projects/1e37cda278c24710a39cd0786c79a269 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:43.687403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.687889 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.688002 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.707262 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:43.708441 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:43.724736 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:43.767803 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1324/2614] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:43.775669 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.775951 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.776222 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.795962 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:43.797150 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:43.804778 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1291/2615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:43.809507 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:43.819137 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f2b0a-dd9a-4ebd-9692-50db85c0db79 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:43.823738 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:43.824335 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.824617 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.824893 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] PATH_INFO: `/v3/domains` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.825535 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81745) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 26 11:37:43.825840 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:43.827467 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:43.838859 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1325/2616] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Oct 26 11:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 26 11:37:43.842865 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:43.851793 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95a15f-fab7-4de4-a024-0f6313fe3082 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:43.856762 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:43.857342 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:43.857615 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:43.857879 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:43.858498 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:43.860404 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:43.944366 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1292/2617] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Oct 26 11:37:43 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 26 11:37:43.948454 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:43.958446 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d58e6-4b28-4ff5-a35a-9f8a169f4280 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:44.013767 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:44.014299 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.014579 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.014871 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.015500 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:44.017403 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:44.091841 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1326/2618] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Oct 26 11:37:43 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 26 11:37:44.097449 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:44.108726 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f56b2-20d5-40ae-9eb5-a0f6646682b4 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:44.113356 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:44.113954 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.114237 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.114513 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.115213 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:44.116797 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:44.126539 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1293/2619] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:37:44.130321 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:44.140498 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d31796-ee12-4590-9cc3-2c0890ff34a9 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:44.145069 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:44.145661 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.145943 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.146219 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.146898 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:44.148475 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b1342-27a1-4745-802e-9c2f7a5cdfb4 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:44.159336 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1327/2620] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Oct 26 11:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 26 11:37:44.163060 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:44.173395 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedebd84-3001-4927-9f65-4d6af587c898 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:44.177903 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:44.178557 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.178860 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.179140 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] PATH_INFO: `/v3/projects/af20f980546143b18a48f4a2a278f673/users/d8bea524038a46df834ad5a7395c8ab5/roles/188e9c2030b54ebaadc92e64b3250166` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.193155 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af20f980546143b18a48f4a2a278f673, user_id=d8bea524038a46df834ad5a7395c8ab5, role_id=188e9c2030b54ebaadc92e64b3250166)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:44.195830 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:44.280787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1294/2621] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Oct 26 11:37:44 2022] PUT /identity/v3/projects/af20f980546143b18a48f4a2a278f673/users/d8bea524038a46df834ad5a7395c8ab5/roles/188e9c2030b54ebaadc92e64b3250166 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:37:44.291101 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] REQUEST_METHOD: `POST` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.291940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.291940 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.318704 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.auth.core [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] MFA Rules not processed for user `d8bea524038a46df834ad5a7395c8ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81745) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:37:44.319944 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:44.382030 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1328/2622] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:37:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:37:44.393407 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:37:44.395112 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:37:44.417082 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:44.452123 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323cb7-404d-4176-aa1d-56d791464b87 admin admin] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:37:44.455674 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b32b5787164f1ba63e5e1e53f78174', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:37:44.456334 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] REQUEST_METHOD: `GET` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:37:44.456612 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:37:44.456885 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:37:44.465500 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:37:44.466952 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f4b31-9e96-4844-8026-f40d23c69211 service cinder] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:37:44.475821 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1295/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:37:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:37:54.379235 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:37:54.491720 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=17 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:38:07.271801 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:38:07.333725 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d3a63-890c-43b5-bf06-bf90bebda828 None None] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:38:07.338252 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:38:07.338902 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:38:07.339180 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:38:07.339447 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] PATH_INFO: `/v3/users/d8bea524038a46df834ad5a7395c8ab5` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:38:07.340554 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bea524038a46df834ad5a7395c8ab5)` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:38:07.343152 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:38:07.503800 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bea524038a46df834ad5a7395c8ab5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:07.510493 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bea524038a46df834ad5a7395c8ab5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:07.511145 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bea524038a46df834ad5a7395c8ab5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:07.581506 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bea524038a46df834ad5a7395c8ab5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:07.582111 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.notifications [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bea524038a46df834ad5a7395c8ab5'} {{(pid=81745) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:07.599202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1329/2624] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Oct 26 11:38:07 2022] DELETE /identity/v3/users/d8bea524038a46df834ad5a7395c8ab5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 26 11:38:07.603787 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] REQUEST_METHOD: `POST` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:38:07.604337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:38:07.604337 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:38:07.624162 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.auth.core [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] MFA Rules not processed for user `e3c50bbb7acb4f91ac247acc31363261`. Rule list: `[]` (Enabled: `True`). {{(pid=81744) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 26 11:38:07.625360 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:38:07.683273 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1296/2625] 10.222.0.42 () {58 vars in 958 bytes} [Wed Oct 26 11:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 26 11:38:07.693605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:38:07.695363 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:38:07.754647 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce7804-e2d0-4619-b149-64b3b6ed6c30 admin admin] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:38:07.758296 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '822597c72e114b549c4b51c2aeaf63d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:38:07.758923 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:38:07.759202 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:38:07.759468 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:38:07.768620 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:38:07.770027 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:38:07.779834 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1330/2626] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 26 11:38:08.193907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] Authenticating user token {{(pid=81744) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:38:08.195402 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.fernet_utils [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81744) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:38:08.207907 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15cd045-861b-44ab-a5b0-c888789345a9 None None] Validating token access rules against request {{(pid=81744) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:38:08.212515 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c50bbb7acb4f91ac247acc31363261', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d130249a55c4bbea044091c5d05b944', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81744) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:38:08.213245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:38:08.213414 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:38:08.213770 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] PATH_INFO: `/v3/projects/af20f980546143b18a48f4a2a278f673` {{(pid=81744) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:38:08.220368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af20f980546143b18a48f4a2a278f673)` {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:38:08.220368 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] RBAC: Authorization granted {{(pid=81744) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:38:08.318385 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af20f980546143b18a48f4a2a278f673', 'request_id': 'req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:08.325245 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af20f980546143b18a48f4a2a278f673', 'request_id': 'req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:08.330773 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af20f980546143b18a48f4a2a278f673', 'request_id': 'req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f'} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:08.414922 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] The token cache is being invalidate because project af20f980546143b18a48f4a2a278f673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81744) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 26 11:38:08.415211 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:08.415825 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG keystone.notifications [None req-9ed0cf9a-fc89-41f4-a6d9-a158677a4a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81744) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 26 11:38:08.417285 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: [pid: 81744|app: 0|req: 1297/2627] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Oct 26 11:38:08 2022] DELETE /identity/v3/projects/af20f980546143b18a48f4a2a278f673 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 26 11:38:17.775205 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:38:18.411714 dv-cinder-815582-18-iscsi devstack@keystone.service[81744]: DEBUG dbcounter [-] [81744] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81744) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 26 11:42:27.672001 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] Authenticating user token {{(pid=81745) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 26 11:42:27.673436 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:42:27.743229 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bcd1f-8613-441b-9cae-222ea05e6ef1 service neutron] Validating token access rules against request {{(pid=81745) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 26 11:42:27.800917 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a26f6a0712462996a17c7fe00a68ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '398b5b4391a3473484fc6044da2bc128', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81745) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 26 11:42:27.801515 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] REQUEST_METHOD: `GET` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 26 11:42:27.801792 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] SCRIPT_NAME: `/identity` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 26 11:42:27.802062 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81745) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 26 11:42:27.803785 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.fernet_utils [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81745) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 26 11:42:27.815264 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 26 11:42:27.816838 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6488b056-8e42-43cc-b516-d45d1967acdd service placement] RBAC: Authorization granted {{(pid=81745) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 26 11:42:27.826605 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: [pid: 81745|app: 0|req: 1331/2628] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Oct 26 11:42:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 26 11:42:37.822309 dv-cinder-815582-18-iscsi devstack@keystone.service[81745]: DEBUG dbcounter [-] [81745] Writing DB stats keystone:SELECT=58 {{(pid=81745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}